AI collection suggestions, pick-what-to-do runs, and a retry for bad replies
- Runs can now do any mix of: tags & descriptions, clearer names, and collection suggestions. A run without tags doesn't mark models as tagged. - Collection ideas reuse your collections and earlier ideas where they fit and wait on the AI page, grouped by name, until you create/add or dismiss them. Dismissed ideas aren't suggested for that model again; spelling variants group together under the capitalised spelling. - A reply that isn't the JSON asked for is asked again once; both requests count towards the run's tokens and cost. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
This commit is contained in:
dlawler489
parent
2076717447
commit
9d7a978f03
9 changed files with 416 additions and 66 deletions
|
|
@ -23,7 +23,7 @@ A self-hosted web app for organizing your 3D-printing files (STL, 3MF, OBJ, FBX,
|
|||
- **Notes** — free-form notes per model, saved as you type.
|
||||
- **Search, filters & sorting** — search across names, descriptions, notes, tags, file names and designers; filter by format, source and license; include *or exclude* tags (match any/all); sort and group by source or format. Press <kbd>/</kbd> to search.
|
||||
- **Duplicate finder** — every file gets a SHA-256 content hash plus (for STL/OBJ/3MF) a geometry fingerprint, computed in the background. Models sharing files are grouped as *exact copies* or *same shape* (renamed, re-exported, converted — or mirrored, so check those), with a recommended copy to keep. Keeping one merges the others' tags, collections and notes into it and moves them to the trash; groups can be dismissed as "not duplicates".
|
||||
- **AI tags & descriptions** — bring your own key for DeepSeek (the default), OpenRouter, OpenAI or any OpenAI-compatible server (Ollama, LM Studio). Tag the whole library, only models not yet tagged, or a single model from its page. Each model's name, file names and (optionally) thumbnail are sent; tags are only ever added and descriptions only filled where empty unless you ask. Optionally it also gives models with unclear names (file-name leftovers, typos, site boilerplate) a clearer one; the old name is kept and can be restored with Undo on the model page. Runs show live progress, tokens and cost, can be stopped and resumed, stop at an optional spending limit, and pause (rather than keep spending) if the server restarts. "Save & test connection" in Settings checks the key, model and whether it accepts images.
|
||||
- **AI tags & descriptions** — bring your own key for DeepSeek (the default), OpenRouter, OpenAI or any OpenAI-compatible server (Ollama, LM Studio). Tag the whole library, only models not yet tagged, or a single model from its page. Each model's name, file names and (optionally) thumbnail are sent; tags are only ever added and descriptions only filled where empty unless you ask. Optionally it also gives models with unclear names (file-name leftovers, typos, site boilerplate) a clearer one; the old name is kept and can be restored with Undo on the model page. It can also suggest collections (reusing yours where they fit): suggestions wait on the AI page, grouped by name, until you create/add or dismiss them. Each run can do any mix of tags & descriptions, names and collections, and a reply that isn't the JSON asked for is retried once. Runs show live progress, tokens and cost, can be stopped and resumed, stop at an optional spending limit, and pause (rather than keep spending) if the server restarts. "Save & test connection" in Settings checks the key, model and whether it accepts images.
|
||||
- **Search by meaning** — switch the search box to *Meaning* to find models by what they are ("wall hook" finds a "coat hanger bracket"). Uses a small embedding model (bge-small-en-v1.5, ~34 MB) that runs on the server, so it's free and private; the index updates in the background as models change.
|
||||
- **Automatic tags** — strict whole-word rules over names, file names and archive contents (part types, mechanisms, materials, techniques) plus your own "word → tag" rules. Switch it off (optionally removing its tags) or re-apply it any time. Tags you add yourself are never changed; AI and rule tags are marked on the model page.
|
||||
- **Trash** — deleting moves a model to the trash; restore it, delete it forever, or empty the trash (only then are files removed from disk).
|
||||
|
|
|
|||
|
|
@ -214,3 +214,14 @@ CREATE TABLE IF NOT EXISTS tag_rules (
|
|||
ALTER TABLE models ADD COLUMN IF NOT EXISTS original_name TEXT;
|
||||
ALTER TABLE ai_run_items ADD COLUMN IF NOT EXISTS renamed_from TEXT;
|
||||
ALTER TABLE ai_run_items ADD COLUMN IF NOT EXISTS renamed_to TEXT;
|
||||
|
||||
-- Collections the AI suggested for a model, waiting for you to accept or dismiss.
|
||||
-- Dismissed ones are kept so the same idea isn't suggested for that model again.
|
||||
CREATE TABLE IF NOT EXISTS collection_suggestions (
|
||||
model_id INTEGER NOT NULL REFERENCES models(id) ON DELETE CASCADE,
|
||||
name TEXT NOT NULL,
|
||||
status TEXT NOT NULL DEFAULT 'pending', -- pending | accepted | dismissed
|
||||
created_at TIMESTAMPTZ NOT NULL DEFAULT now()
|
||||
);
|
||||
CREATE UNIQUE INDEX IF NOT EXISTS idx_collection_suggestions_model_name ON collection_suggestions(model_id, lower(name));
|
||||
ALTER TABLE ai_run_items ADD COLUMN IF NOT EXISTS collections TEXT[];
|
||||
|
|
|
|||
133
lib/ai.js
133
lib/ai.js
|
|
@ -1,6 +1,6 @@
|
|||
// AI tags and descriptions through any OpenAI-compatible chat API: DeepSeek by
|
||||
// default, or OpenRouter, OpenAI, or a local server such as Ollama or LM Studio.
|
||||
// The API key stays on the server.
|
||||
// AI tags, descriptions, names and collection ideas through any OpenAI-compatible
|
||||
// chat API: DeepSeek by default, or OpenRouter, OpenAI, or a local server such as
|
||||
// Ollama or LM Studio. The API key stays on the server.
|
||||
|
||||
const PROVIDERS = {
|
||||
deepseek: { label: 'DeepSeek', baseUrl: 'https://api.deepseek.com', model: 'deepseek-flash', prices: { input: 0.3, output: 1.2 } },
|
||||
|
|
@ -9,13 +9,18 @@ const PROVIDERS = {
|
|||
custom: { label: 'Other (OpenAI-compatible)', baseUrl: '', model: '' },
|
||||
};
|
||||
|
||||
const SYSTEM_PROMPT = `You catalogue 3D-printable models in a maker's library. Reply with JSON only, shaped {"tags": ["..."], "description": "..."}.
|
||||
tags: 3 to 8 short lowercase tags someone would search for: what the object is, what it's for, its theme or style, and print traits (print-in-place, articulated, multicolor, needs supports) only when evident. Prefer singular nouns. When a tag already used in the library fits, use it instead of a new synonym. Never use generic tags like "3d print", "model", "stl" or "3mf".
|
||||
description: one or two plain sentences on what the object is and what it's for. Don't invent sizes, print settings or claims.`;
|
||||
// The system prompt is assembled from the parts a run asks for.
|
||||
const BASE_PROMPT = `You catalogue 3D-printable models in a maker's library. Reply with JSON only, as one object containing exactly the fields described below.`;
|
||||
|
||||
const DESCRIBE_PROMPT = `
|
||||
"tags": 3 to 8 short lowercase tags someone would search for: what the object is, what it's for, its theme or style, and print traits (print-in-place, articulated, multicolor, needs supports) only when evident. Prefer singular nouns. When a tag already used in the library fits, use it instead of a new synonym. Never use generic tags like "3d print", "model", "stl" or "3mf".
|
||||
"description": one or two plain sentences on what the object is and what it's for. Don't invent sizes, print settings or claims.`;
|
||||
|
||||
// Added to the prompt only when a run may rename models.
|
||||
const NAME_PROMPT = `
|
||||
Also include "name": the model's name as a clear title. If the current name already says clearly what the model is, return it exactly as it is. Otherwise write a short title-case name (under 60 characters) saying what it is, using the picture and details: drop file-name leftovers (underscores, stray spaces, version numbers like "v4", "final", file extensions), fix typos, and remove site boilerplate like "Free 3D Print Model - MakerWorld". Keep designer, character or product names that are part of what it is.`;
|
||||
"name": the model's name as a clear title. If the current name already says clearly what the model is, return it exactly as it is. Otherwise write a short title-case name (under 60 characters) saying what it is, using the picture and details: drop file-name leftovers (underscores, stray spaces, version numbers like "v4", "final", file extensions), fix typos, and remove site boilerplate like "Free 3D Print Model - MakerWorld". Keep designer, character or product names that are part of what it is.`;
|
||||
|
||||
const COLLECTIONS_PROMPT = `
|
||||
"collections": 0 to 3 collections this model belongs in: broad groups someone would browse a library by, such as "Bookends & Book Holders", "Desk Organizers", "Pokémon" or "Halloween", never a name for this one model. When one of the library's existing collections fits, use its name exactly as written. Return [] when nothing fits well.`;
|
||||
|
||||
const GENERIC_TAGS = new Set([
|
||||
'3d print', '3d printing', '3d printed', '3d printable', '3d model', 'model', 'print', 'printable', 'file',
|
||||
|
|
@ -30,6 +35,9 @@ class AiError extends Error {
|
|||
}
|
||||
}
|
||||
|
||||
// The provider answered, but not with the JSON asked for.
|
||||
class ReplyError extends Error {}
|
||||
|
||||
const sleep = (ms) => new Promise((resolve) => setTimeout(resolve, ms));
|
||||
|
||||
// Settings values (ai_provider, ai_base_url, …) -> a client config, filling the
|
||||
|
|
@ -85,12 +93,25 @@ function modelSummary(model, knownTags = []) {
|
|||
return lines.join('\n');
|
||||
}
|
||||
|
||||
function buildMessages(model, { image = null, knownTags = [], suggestName = false } = {}) {
|
||||
const text = modelSummary(model, knownTags);
|
||||
// Which fields to ask for: describe (tags + description, the default), a clearer
|
||||
// name, and collection ideas.
|
||||
function buildMessages(model, {
|
||||
image = null, knownTags = [], describe = true, suggestName = false, suggestCollections = false, collectionNames = [],
|
||||
} = {}) {
|
||||
let text = modelSummary(model, describe ? knownTags : []);
|
||||
if (suggestCollections) {
|
||||
text += collectionNames.length
|
||||
? `\nExisting collections in this library: ${collectionNames.join(', ')}`
|
||||
: '\nThis library has no collections yet.';
|
||||
}
|
||||
const system = BASE_PROMPT
|
||||
+ (describe ? DESCRIBE_PROMPT : '')
|
||||
+ (suggestName ? NAME_PROMPT : '')
|
||||
+ (suggestCollections ? COLLECTIONS_PROMPT : '');
|
||||
const content = image
|
||||
? [{ type: 'text', text }, { type: 'image_url', image_url: { url: image } }]
|
||||
: text;
|
||||
return [{ role: 'system', content: SYSTEM_PROMPT + (suggestName ? NAME_PROMPT : '') }, { role: 'user', content }];
|
||||
return [{ role: 'system', content: system }, { role: 'user', content }];
|
||||
}
|
||||
|
||||
function normalizeTags(value) {
|
||||
|
|
@ -110,13 +131,13 @@ function normalizeTags(value) {
|
|||
}
|
||||
|
||||
const comparable = (s) => String(s || '').toLowerCase().replace(/[^\p{L}\p{N}]+/gu, '');
|
||||
const stripQuotes = (s) => String(s).replace(/^["'“”‘’]+|["'“”‘’]+$/g, '');
|
||||
|
||||
// A suggested name, tidied; null when there isn't one or it's the current name
|
||||
// written differently.
|
||||
function cleanName(value, currentName) {
|
||||
if (typeof value !== 'string') return null;
|
||||
const name = value
|
||||
.replace(/^["'“”‘’]+|["'“”‘’]+$/g, '')
|
||||
const name = stripQuotes(value)
|
||||
.replace(/\.(stl|3mf|obj|fbx|step|stp|3dm|gcode|zip|7z|rar)$/i, '')
|
||||
.replace(/\s+/g, ' ')
|
||||
.trim()
|
||||
|
|
@ -125,10 +146,22 @@ function cleanName(value, currentName) {
|
|||
return name && comparable(name) !== comparable(currentName) ? name : null;
|
||||
}
|
||||
|
||||
// The model's reply -> { tags, description, name }. Tolerates code fences or
|
||||
// prose around the JSON. name is only read when { currentName } is given (the
|
||||
// prompt asked for one).
|
||||
// Up to 3 collection names, tidied, without repeats that differ only in case or punctuation.
|
||||
function normalizeCollections(value) {
|
||||
const list = Array.isArray(value) ? value : typeof value === 'string' ? value.split(',') : [];
|
||||
const names = [];
|
||||
for (const raw of list) {
|
||||
const name = stripQuotes(raw).replace(/\s+/g, ' ').trim().slice(0, 60).trim();
|
||||
if (name && !names.some((n) => comparable(n) === comparable(name))) names.push(name);
|
||||
}
|
||||
return names.slice(0, 3);
|
||||
}
|
||||
|
||||
// The model's reply -> { tags, description, name, collections }. Tolerates code
|
||||
// fences or prose around the JSON. Only the fields asked for are read: name when
|
||||
// { currentName } is given, collections when { collections: true }.
|
||||
function parseReply(content, options = {}) {
|
||||
const { describe = true, collections = false } = options;
|
||||
const text = String(content || '');
|
||||
const start = text.indexOf('{');
|
||||
const end = text.lastIndexOf('}');
|
||||
|
|
@ -136,12 +169,16 @@ function parseReply(content, options = {}) {
|
|||
if (start >= 0 && end > start) {
|
||||
try { data = JSON.parse(text.slice(start, end + 1)); } catch { /* handled below */ }
|
||||
}
|
||||
if (!data || typeof data !== 'object') throw new Error('The AI reply wasn’t the JSON asked for');
|
||||
const tags = normalizeTags(data.tags);
|
||||
const description = typeof data.description === 'string' ? data.description.trim().slice(0, 600) : '';
|
||||
if (!tags.length && !description) throw new Error('The AI reply had no tags or description');
|
||||
const name = Object.prototype.hasOwnProperty.call(options, 'currentName') ? cleanName(data.name, options.currentName) : null;
|
||||
return { tags, description, name };
|
||||
if (!data || typeof data !== 'object') throw new ReplyError('The AI reply wasn’t the JSON asked for');
|
||||
const tags = describe ? normalizeTags(data.tags) : [];
|
||||
const description = describe && typeof data.description === 'string' ? data.description.trim().slice(0, 600) : '';
|
||||
if (describe && !tags.length && !description) throw new ReplyError('The AI reply had no tags or description');
|
||||
return {
|
||||
tags,
|
||||
description,
|
||||
name: Object.prototype.hasOwnProperty.call(options, 'currentName') ? cleanName(data.name, options.currentName) : null,
|
||||
collections: collections ? normalizeCollections(data.collections) : [],
|
||||
};
|
||||
}
|
||||
|
||||
function usageOf(usage) {
|
||||
|
|
@ -160,6 +197,14 @@ function costOf(usage, prices) {
|
|||
return (prompt * prices.input + completion * prices.output) / 1e6;
|
||||
}
|
||||
|
||||
// Adds one response's usage to a running total, so retried requests are still counted.
|
||||
function addUsage(total, usage) {
|
||||
const { prompt, completion } = usageOf(usage);
|
||||
total.prompt_tokens += prompt;
|
||||
total.completion_tokens += completion;
|
||||
if (usage && typeof usage.cost === 'number') total.cost = (total.cost || 0) + usage.cost;
|
||||
}
|
||||
|
||||
async function chat(config, body, { timeoutMs = 90000 } = {}) {
|
||||
const headers = { 'Content-Type': 'application/json' };
|
||||
if (config.apiKey) headers.Authorization = `Bearer ${config.apiKey}`;
|
||||
|
|
@ -187,18 +232,27 @@ async function chat(config, body, { timeoutMs = 90000 } = {}) {
|
|||
return data;
|
||||
}
|
||||
|
||||
// Tags and a description for one model. Adapts to what the model supports: if it
|
||||
// refuses JSON mode or images, the request is retried without them
|
||||
// (imageRejected tells the caller to stop sending images). Rate limits and
|
||||
// server errors are retried with backoff.
|
||||
async function describeModel(config, model, { image = null, knownTags = [], suggestName = false } = {}) {
|
||||
// Asks about one model: tags and a description, and optionally a clearer name and
|
||||
// collection ideas. Adapts to what the model supports: if it refuses JSON mode or
|
||||
// images, the request is retried without them (imageRejected tells the caller to
|
||||
// stop sending images). A reply that isn't the JSON asked for is asked again once.
|
||||
// Rate limits and server errors are retried with backoff.
|
||||
async function describeModel(config, model, {
|
||||
image = null, knownTags = [], describe = true, suggestName = false, suggestCollections = false, collectionNames = [],
|
||||
} = {}) {
|
||||
let useImage = Boolean(image);
|
||||
let useJson = true;
|
||||
let imageRejected = false;
|
||||
let replyRetried = false;
|
||||
const spent = { prompt_tokens: 0, completion_tokens: 0, cost: null };
|
||||
const want = { describe, collections: suggestCollections, ...(suggestName ? { currentName: model.name } : {}) };
|
||||
|
||||
for (let attempt = 0; ; attempt++) {
|
||||
const body = {
|
||||
model: config.model,
|
||||
messages: buildMessages(model, { image: useImage ? image : null, knownTags, suggestName }),
|
||||
messages: buildMessages(model, {
|
||||
image: useImage ? image : null, knownTags, describe, suggestName, suggestCollections, collectionNames,
|
||||
}),
|
||||
temperature: 0.2,
|
||||
max_tokens: 800,
|
||||
};
|
||||
|
|
@ -206,13 +260,22 @@ async function describeModel(config, model, { image = null, knownTags = [], sugg
|
|||
if (config.provider === 'openrouter') body.usage = { include: true };
|
||||
try {
|
||||
const data = await chat(config, body);
|
||||
const message = data.choices[0].message || {};
|
||||
addUsage(spent, data.usage);
|
||||
let reply;
|
||||
try {
|
||||
reply = parseReply((data.choices[0].message || {}).content, want);
|
||||
} catch (err) {
|
||||
// Usually a one-off: ask once more before giving up on this model.
|
||||
if (err instanceof ReplyError && !replyRetried) { replyRetried = true; continue; }
|
||||
throw err;
|
||||
}
|
||||
return {
|
||||
...parseReply(message.content, suggestName ? { currentName: model.name } : {}),
|
||||
usage: usageOf(data.usage),
|
||||
cost: costOf(data.usage, config.prices),
|
||||
...reply,
|
||||
usage: usageOf(spent),
|
||||
cost: costOf(spent, config.prices),
|
||||
imageUsed: useImage,
|
||||
imageRejected,
|
||||
replyRetried,
|
||||
};
|
||||
} catch (err) {
|
||||
if (!(err instanceof AiError) || attempt >= 5) throw err;
|
||||
|
|
@ -244,7 +307,7 @@ function fatalMessage(err) {
|
|||
return null;
|
||||
}
|
||||
|
||||
// A rough cost for tagging `count` models, from typical token counts.
|
||||
// A rough cost for `count` models, from typical token counts.
|
||||
function estimate(count, config) {
|
||||
const prompt = count * (config.sendImages ? 1400 : 500);
|
||||
const completion = count * 120;
|
||||
|
|
@ -253,6 +316,6 @@ function estimate(count, config) {
|
|||
}
|
||||
|
||||
module.exports = {
|
||||
PROVIDERS, AiError, configFrom, missingConfig, buildMessages, parseReply, normalizeTags,
|
||||
costOf, describeModel, fatalMessage, estimate,
|
||||
PROVIDERS, AiError, ReplyError, configFrom, missingConfig, buildMessages, parseReply, normalizeTags,
|
||||
normalizeCollections, costOf, describeModel, fatalMessage, estimate,
|
||||
};
|
||||
|
|
|
|||
|
|
@ -1,5 +1,5 @@
|
|||
// AI tagging runs: a queue of models tagged a few at a time. Progress, tokens and
|
||||
// cost live in the database, so a run can be watched, stopped and resumed, and a
|
||||
// AI runs: a queue of models handled a few at a time. Progress, tokens and cost
|
||||
// live in the database, so a run can be watched, stopped and resumed, and a
|
||||
// server restart pauses it instead of spending on its own.
|
||||
const fs = require('fs');
|
||||
const path = require('path');
|
||||
|
|
@ -47,10 +47,20 @@ async function entryNames(model) {
|
|||
return rows.map((r) => r.path.split('/').pop()).filter(Boolean);
|
||||
}
|
||||
|
||||
// options: describe (tags + descriptions, default on), overwriteDescriptions,
|
||||
// renameUnclear, suggestCollections, maxCost.
|
||||
async function createRun(modelIds, options = {}) {
|
||||
const config = await resolveConfig();
|
||||
const problem = ai.missingConfig(config);
|
||||
if (problem) throw httpError(problem, 400);
|
||||
const tasks = {
|
||||
describe: options.describe !== false,
|
||||
renameUnclear: Boolean(options.renameUnclear),
|
||||
suggestCollections: Boolean(options.suggestCollections),
|
||||
};
|
||||
if (!tasks.describe && !tasks.renameUnclear && !tasks.suggestCollections) {
|
||||
throw httpError('Choose at least one thing for the AI to do', 400);
|
||||
}
|
||||
const ids = [...new Set((modelIds || []).map(Number).filter(Number.isInteger))];
|
||||
const maxCost = Number(options.maxCost) > 0 ? Number(options.maxCost) : null;
|
||||
|
||||
|
|
@ -58,8 +68,8 @@ async function createRun(modelIds, options = {}) {
|
|||
const { rows: [created] } = await client.query(
|
||||
`INSERT INTO ai_runs (provider, model, options) VALUES ($1, $2, $3) RETURNING id`,
|
||||
[config.label, config.model, {
|
||||
overwriteDescriptions: Boolean(options.overwriteDescriptions),
|
||||
renameUnclear: Boolean(options.renameUnclear),
|
||||
...tasks,
|
||||
overwriteDescriptions: tasks.describe && Boolean(options.overwriteDescriptions),
|
||||
maxCost,
|
||||
pricesKnown: config.prices.input != null || config.provider === 'openrouter',
|
||||
}]);
|
||||
|
|
@ -80,7 +90,8 @@ async function finish(runId, status, error = null) {
|
|||
[runId, status, error]);
|
||||
}
|
||||
|
||||
async function tagOne(runId, modelId, config, { images, knownTags, overwrite, rename }) {
|
||||
async function tagOne(runId, modelId, config, context) {
|
||||
const { images, knownTags, collectionNames, tasks } = context;
|
||||
const model = await store.getModel(modelId);
|
||||
if (!model || model.deleted_at) {
|
||||
await db.query(`UPDATE ai_run_items SET status = 'error', error = 'The model was deleted', updated_at = now()
|
||||
|
|
@ -90,18 +101,32 @@ async function tagOne(runId, modelId, config, { images, knownTags, overwrite, re
|
|||
}
|
||||
try {
|
||||
model.entryNames = await entryNames(model);
|
||||
const result = await ai.describeModel(config, model,
|
||||
{ image: images ? thumbnailDataUrl(model) : null, knownTags, suggestName: rename });
|
||||
const name = rename ? result.name : null;
|
||||
await store.applyAiResult(modelId, { tags: result.tags, description: result.description, overwriteDescription: overwrite, name });
|
||||
await db.query(`UPDATE ai_run_items SET status = 'done', tags = $3, error = NULL, renamed_from = $4, renamed_to = $5,
|
||||
updated_at = now() WHERE run_id = $1 AND model_id = $2`,
|
||||
[runId, modelId, result.tags, name ? model.name : null, name]);
|
||||
const result = await ai.describeModel(config, model, {
|
||||
image: images ? thumbnailDataUrl(model) : null,
|
||||
knownTags,
|
||||
describe: tasks.describe,
|
||||
suggestName: tasks.renameUnclear,
|
||||
suggestCollections: tasks.suggestCollections,
|
||||
collectionNames,
|
||||
});
|
||||
const name = tasks.renameUnclear ? result.name : null;
|
||||
await store.applyAiResult(modelId, {
|
||||
tags: result.tags,
|
||||
description: result.description,
|
||||
overwriteDescription: tasks.overwriteDescriptions,
|
||||
name,
|
||||
markTagged: tasks.describe,
|
||||
});
|
||||
const suggested = tasks.suggestCollections ? await store.suggestCollections(modelId, result.collections) : [];
|
||||
await db.query(
|
||||
`UPDATE ai_run_items SET status = 'done', tags = $3, error = NULL, renamed_from = $4, renamed_to = $5,
|
||||
collections = $6, updated_at = now() WHERE run_id = $1 AND model_id = $2`,
|
||||
[runId, modelId, tasks.describe ? result.tags : null, name ? model.name : null, name, suggested]);
|
||||
await db.query(
|
||||
`UPDATE ai_runs SET done = done + 1, prompt_tokens = prompt_tokens + $2,
|
||||
completion_tokens = completion_tokens + $3, cost_usd = cost_usd + $4 WHERE id = $1`,
|
||||
[runId, result.usage.prompt, result.usage.completion, result.cost || 0]);
|
||||
return { ok: true, imageRejected: result.imageRejected };
|
||||
return { ok: true, imageRejected: result.imageRejected, suggested: suggested.length > 0 };
|
||||
} catch (err) {
|
||||
const fatal = ai.fatalMessage(err);
|
||||
if (fatal) return { ok: false, fatal }; // left pending, so resuming retries it
|
||||
|
|
@ -121,30 +146,41 @@ async function processRun(runId) {
|
|||
let images = config.sendImages;
|
||||
let failuresInARow = 0;
|
||||
let lastError = null;
|
||||
let collectionNames = null;
|
||||
|
||||
for (;;) {
|
||||
const { rows: [run] } = await db.query('SELECT status, options, cost_usd FROM ai_runs WHERE id = $1', [runId]);
|
||||
if (!run || run.status !== 'running') return; // stopped, or deleted
|
||||
const { maxCost, overwriteDescriptions, renameUnclear } = run.options || {};
|
||||
const options = run.options || {};
|
||||
// Runs from before these options existed only tagged and described.
|
||||
const tasks = {
|
||||
describe: options.describe !== false,
|
||||
overwriteDescriptions: Boolean(options.overwriteDescriptions),
|
||||
renameUnclear: Boolean(options.renameUnclear),
|
||||
suggestCollections: Boolean(options.suggestCollections),
|
||||
};
|
||||
const { rows: items } = await db.query(
|
||||
`SELECT model_id FROM ai_run_items WHERE run_id = $1 AND status = 'pending' ORDER BY model_id LIMIT $2`,
|
||||
[runId, CONCURRENCY]);
|
||||
if (!items.length) { await finish(runId, 'done'); return; }
|
||||
// Checked between batches, so a run can go over by up to one batch.
|
||||
if (maxCost && Number(run.cost_usd) >= maxCost) {
|
||||
await finish(runId, 'stopped', `Stopped at your $${maxCost} spending limit`);
|
||||
if (options.maxCost && Number(run.cost_usd) >= options.maxCost) {
|
||||
await finish(runId, 'stopped', `Stopped at your $${options.maxCost} spending limit`);
|
||||
return;
|
||||
}
|
||||
// Existing collections and earlier ideas, so suggestions reuse the same names.
|
||||
if (tasks.suggestCollections && collectionNames === null) collectionNames = await store.collectionNameHints();
|
||||
|
||||
const results = await Promise.all(items.map((item) =>
|
||||
tagOne(runId, item.model_id, config, { images, knownTags, overwrite: overwriteDescriptions, rename: renameUnclear })));
|
||||
// New tags and descriptions change a model's search text, so refresh the index.
|
||||
tagOne(runId, item.model_id, config, { images, knownTags, collectionNames: collectionNames || [], tasks })));
|
||||
// New tags, descriptions and names change a model's search text, so refresh the index.
|
||||
if (results.some((r) => r.ok)) fingerprint.kick();
|
||||
if (results.some((r) => r.suggested)) collectionNames = null;
|
||||
for (const r of results) {
|
||||
if (r.imageRejected && images) {
|
||||
images = false;
|
||||
await db.query('UPDATE ai_runs SET note = $2 WHERE id = $1',
|
||||
[runId, `${config.model} doesn't accept images, so models are tagged from their names and files.`]);
|
||||
[runId, `${config.model} doesn't accept images, so models are judged from their names and files.`]);
|
||||
}
|
||||
if (r.fatal) { await finish(runId, 'failed', r.fatal); return; }
|
||||
if (r.ok) failuresInARow = 0;
|
||||
|
|
@ -197,8 +233,8 @@ async function getRun(id) {
|
|||
const { rows: [run] } = await db.query('SELECT * FROM ai_runs WHERE id = $1', [id]);
|
||||
if (!run) return null;
|
||||
const { rows: recent } = await db.query(
|
||||
`SELECT i.model_id, m.name, i.status, i.tags, i.error, i.renamed_from, i.renamed_to FROM ai_run_items i
|
||||
LEFT JOIN models m ON m.id = i.model_id
|
||||
`SELECT i.model_id, m.name, i.status, i.tags, i.error, i.renamed_from, i.renamed_to, i.collections
|
||||
FROM ai_run_items i LEFT JOIN models m ON m.id = i.model_id
|
||||
WHERE i.run_id = $1 AND i.status <> 'pending' ORDER BY i.updated_at DESC LIMIT 12`, [id]);
|
||||
return { ...shapeRun(run), recent };
|
||||
}
|
||||
|
|
|
|||
101
lib/store.js
101
lib/store.js
|
|
@ -347,7 +347,8 @@ async function removeTagsFrom(source) {
|
|||
// AI tagging: adds the suggested tags (never removes any), fills in the
|
||||
// description if it's empty (or replaces it when asked to), and renames the model
|
||||
// when given a name, keeping the name it had before AI first renamed it.
|
||||
async function applyAiResult(modelId, { tags, description, overwriteDescription, name = null }) {
|
||||
// markTagged records that the model has had AI tags and a description.
|
||||
async function applyAiResult(modelId, { tags, description, overwriteDescription, name = null, markTagged = true }) {
|
||||
await db.transaction(async (client) => {
|
||||
for (const name of tags) {
|
||||
const id = await tagId(client, name);
|
||||
|
|
@ -355,14 +356,107 @@ async function applyAiResult(modelId, { tags, description, overwriteDescription,
|
|||
[modelId, id]);
|
||||
}
|
||||
await client.query(
|
||||
`UPDATE models SET ai_tagged_at = now(),
|
||||
`UPDATE models SET ai_tagged_at = CASE WHEN $5::boolean THEN now() ELSE ai_tagged_at END,
|
||||
description = CASE WHEN $2::text <> '' AND ($3::boolean OR btrim(description) = '') THEN $2 ELSE description END,
|
||||
original_name = CASE WHEN $4::text IS NOT NULL THEN COALESCE(original_name, name) ELSE original_name END,
|
||||
name = COALESCE($4::text, name)
|
||||
WHERE id = $1`, [modelId, description || '', Boolean(overwriteDescription), name]);
|
||||
WHERE id = $1`, [modelId, description || '', Boolean(overwriteDescription), name, Boolean(markTagged)]);
|
||||
});
|
||||
}
|
||||
|
||||
// ---------------------------------------------------------------- collection suggestions
|
||||
// Records the AI's collection ideas for a model. A name matching an existing
|
||||
// collection or earlier idea takes its spelling, so ideas group together; ideas
|
||||
// the model already belongs to, or that you dismissed for it, are skipped.
|
||||
// Returns the names recorded.
|
||||
async function suggestCollections(modelId, names) {
|
||||
const recorded = [];
|
||||
for (const raw of names) {
|
||||
const { rows: [known] } = await db.query(
|
||||
`SELECT name FROM (
|
||||
SELECT name, 0 AS rank FROM projects WHERE lower(name) = lower($1)
|
||||
UNION ALL
|
||||
SELECT name, 1 AS rank FROM collection_suggestions WHERE lower(name) = lower($1)
|
||||
) n ORDER BY rank, name = lower(name), name LIMIT 1`, [raw]);
|
||||
// An all-lowercase idea gives way to a capitalised spelling of the same name.
|
||||
const name = known && !(known.name === known.name.toLowerCase() && raw !== raw.toLowerCase()) ? known.name : raw;
|
||||
const { rowCount: member } = await db.query(
|
||||
`SELECT 1 FROM model_projects mp JOIN projects p ON p.id = mp.project_id
|
||||
WHERE mp.model_id = $1 AND lower(p.name) = lower($2)`, [modelId, name]);
|
||||
if (member) continue;
|
||||
const { rowCount } = await db.query(
|
||||
'INSERT INTO collection_suggestions (model_id, name) VALUES ($1, $2) ON CONFLICT DO NOTHING', [modelId, name]);
|
||||
if (rowCount) recorded.push(name);
|
||||
}
|
||||
return recorded;
|
||||
}
|
||||
|
||||
// Collection names the AI should reuse: your collections, then ideas waiting for review.
|
||||
async function collectionNameHints(limit = 100) {
|
||||
const { rows } = await db.query(
|
||||
`SELECT name FROM projects
|
||||
UNION
|
||||
SELECT name FROM collection_suggestions WHERE status = 'pending'
|
||||
ORDER BY name LIMIT $1`, [limit]);
|
||||
return rows.map((r) => r.name);
|
||||
}
|
||||
|
||||
// Waiting ideas grouped by name: [{ name, project_id (if it exists), models: [{ id, name }] }].
|
||||
async function listCollectionSuggestions() {
|
||||
const { rows } = await db.query(`
|
||||
SELECT (array_agg(COALESCE(p.name, s.name) ORDER BY p.id IS NULL, s.name = lower(s.name), s.name))[1] AS name,
|
||||
MIN(p.id) AS project_id,
|
||||
json_agg(DISTINCT jsonb_build_object('id', m.id, 'name', m.name)) AS models
|
||||
FROM collection_suggestions s
|
||||
JOIN models m ON m.id = s.model_id AND m.deleted_at IS NULL
|
||||
LEFT JOIN projects p ON lower(p.name) = lower(s.name)
|
||||
WHERE s.status = 'pending'
|
||||
AND NOT EXISTS (
|
||||
SELECT 1 FROM model_projects mp JOIN projects pp ON pp.id = mp.project_id
|
||||
WHERE mp.model_id = m.id AND lower(pp.name) = lower(s.name))
|
||||
GROUP BY lower(s.name)
|
||||
ORDER BY COUNT(DISTINCT m.id) DESC, 1`);
|
||||
return rows.map((r) => ({ ...r, models: r.models.sort((a, b) => a.name.localeCompare(b.name)) }));
|
||||
}
|
||||
|
||||
// Adds the models an idea was suggested for (or just modelIds) to the collection of
|
||||
// that name, creating it if needed. Returns { project, added }, or null if none waited.
|
||||
async function acceptCollectionSuggestion(name, modelIds = null) {
|
||||
return db.transaction(async (client) => {
|
||||
const { rows: waiting } = await client.query(
|
||||
`SELECT s.model_id FROM collection_suggestions s JOIN models m ON m.id = s.model_id AND m.deleted_at IS NULL
|
||||
WHERE s.status = 'pending' AND lower(s.name) = lower($1) AND ($2::int[] IS NULL OR s.model_id = ANY($2::int[]))`,
|
||||
[name, modelIds]);
|
||||
if (!waiting.length) return null;
|
||||
let { rows: [project] } = await client.query(
|
||||
'SELECT * FROM projects WHERE lower(name) = lower($1) ORDER BY id LIMIT 1', [name]);
|
||||
if (!project) {
|
||||
// Named with the idea's best spelling, however it was asked for.
|
||||
({ rows: [project] } = await client.query(
|
||||
`INSERT INTO projects (name)
|
||||
SELECT COALESCE((SELECT name FROM collection_suggestions WHERE status = 'pending' AND lower(name) = lower($1)
|
||||
ORDER BY name = lower(name), name LIMIT 1), $1)
|
||||
RETURNING *`, [name]));
|
||||
}
|
||||
const ids = waiting.map((w) => w.model_id);
|
||||
for (const id of ids) {
|
||||
await client.query('INSERT INTO model_projects (model_id, project_id) VALUES ($1, $2) ON CONFLICT DO NOTHING', [id, project.id]);
|
||||
}
|
||||
await client.query(
|
||||
`UPDATE collection_suggestions SET status = 'accepted' WHERE lower(name) = lower($1) AND model_id = ANY($2::int[])`,
|
||||
[name, ids]);
|
||||
return { project, added: ids.length };
|
||||
});
|
||||
}
|
||||
|
||||
async function dismissCollectionSuggestion(name, modelIds = null) {
|
||||
const { rowCount } = await db.query(
|
||||
`UPDATE collection_suggestions SET status = 'dismissed'
|
||||
WHERE status = 'pending' AND lower(name) = lower($1) AND ($2::int[] IS NULL OR model_id = ANY($2::int[]))`,
|
||||
[name, modelIds]);
|
||||
return rowCount;
|
||||
}
|
||||
|
||||
// The library's most used tags, so AI suggestions reuse the same words.
|
||||
async function topTags(limit = 150) {
|
||||
const { rows } = await db.query(
|
||||
|
|
@ -541,6 +635,7 @@ module.exports = {
|
|||
duplicateCandidateFiles, matchableCounts, mergeAndTrash, dismissDuplicateGroup, dismissedDuplicateSignatures,
|
||||
getFile, getArchiveEntries, getArchiveEntry, replaceArchiveEntries,
|
||||
getAllTags, replaceRuleTags, removeTagsFrom, applyAiResult, topTags,
|
||||
suggestCollections, collectionNameHints, listCollectionSuggestions, acceptCollectionSuggestion, dismissCollectionSuggestion,
|
||||
getAllProjects, getProject, createProject, updateProject, deleteProject,
|
||||
getSetting, setSetting,
|
||||
countUsers, getUserByEmail, getUserById, createUser, updateUserPassword,
|
||||
|
|
|
|||
|
|
@ -783,6 +783,7 @@ function setupEventListeners() {
|
|||
});
|
||||
fileGrid.addEventListener('change', (e) => {
|
||||
if (e.target.name === 'aiScope') updateAiEstimate();
|
||||
if (e.target.id === 'aiDescribe') document.getElementById('aiOverwrite').disabled = !e.target.checked;
|
||||
if (e.target.id === 'autotagEnabled') handleAutotagToggle(e.target.checked);
|
||||
});
|
||||
document.getElementById('openInSlicerBtn').addEventListener('click', () => {
|
||||
|
|
@ -1374,8 +1375,13 @@ function aiPanelHtml() {
|
|||
<label><input type="radio" name="aiScope" value="all"> All models
|
||||
<span class="count-inline" id="aiAllCount"></span></label>
|
||||
</div>
|
||||
<label><input type="checkbox" id="aiOverwrite"> Replace descriptions that are already filled in</label>
|
||||
<label><input type="checkbox" id="aiRename"> Give models with unclear names a clearer one (undo on the model's page)</label>
|
||||
<fieldset class="ai-tasks">
|
||||
<legend>What to do</legend>
|
||||
<label><input type="checkbox" id="aiDescribe" checked> Add tags and fill in empty descriptions</label>
|
||||
<label class="ai-subtask"><input type="checkbox" id="aiOverwrite"> …and replace descriptions that are already filled in</label>
|
||||
<label><input type="checkbox" id="aiRename"> Give models with unclear names a clearer one (undo on the model's page)</label>
|
||||
<label><input type="checkbox" id="aiCollections"> Suggest collections for you to review</label>
|
||||
</fieldset>
|
||||
<label>Stop when the run has cost
|
||||
<span class="money-input">$<input type="number" id="aiMaxCost" min="0" step="0.01" placeholder="no limit"></span></label>
|
||||
<div class="ai-start-row">
|
||||
|
|
@ -1385,6 +1391,11 @@ function aiPanelHtml() {
|
|||
</form>
|
||||
<div id="aiRuns" class="ai-runs"></div>
|
||||
</section>
|
||||
<section class="ai-card">
|
||||
<div class="ai-card-head"><h3>Suggested collections</h3></div>
|
||||
<p class="form-hint">Ideas from runs with “Suggest collections” ticked. Nothing changes until you accept one.</p>
|
||||
<div id="aiSuggestions" class="form-hint">Loading…</div>
|
||||
</section>
|
||||
<section class="ai-card">
|
||||
<div class="ai-card-head"><h3>Search by meaning</h3></div>
|
||||
<p class="form-hint">Switch the search box to <strong>Meaning</strong> to find models by what they are, so
|
||||
|
|
@ -1409,6 +1420,7 @@ function runHtml(run, detail) {
|
|||
${i.renamed_to ? `<span class="renamed-from">was “${escapeHtml(i.renamed_from)}”</span>` : ''}
|
||||
${i.status === 'done'
|
||||
? (i.tags || []).map((t) => `<span class="tag">${escapeHtml(t)}</span>`).join('')
|
||||
+ ((i.collections || []).length ? `<span class="suggested-collections">→ ${i.collections.map(escapeHtml).join(', ')}</span>` : '')
|
||||
: `<span class="ai-run-error">${escapeHtml(i.error || 'failed')}</span>`}
|
||||
</li>`).join('')}
|
||||
</ul>` : '';
|
||||
|
|
@ -1488,6 +1500,7 @@ async function renderAiPanel() {
|
|||
if (selectedScope !== 'ai') return;
|
||||
fileGrid.innerHTML = aiPanelHtml();
|
||||
renderRulesCard();
|
||||
renderCollectionSuggestions();
|
||||
await refreshAiPanel();
|
||||
updateAiEstimate();
|
||||
}
|
||||
|
|
@ -1520,6 +1533,7 @@ async function refreshAiPanel() {
|
|||
if (changed && !running) {
|
||||
aiState.expectChange = false;
|
||||
loadData(); // the run changed tags and descriptions
|
||||
renderCollectionSuggestions();
|
||||
}
|
||||
if (running || status.search.pending || status.autotag.pending) {
|
||||
aiPoll = setTimeout(refreshAiPanel, running ? 2000 : 4000);
|
||||
|
|
@ -1557,6 +1571,37 @@ function renderAiLive() {
|
|||
}
|
||||
}
|
||||
|
||||
async function renderCollectionSuggestions() {
|
||||
const el = document.getElementById('aiSuggestions');
|
||||
if (!el) return;
|
||||
let groups;
|
||||
try {
|
||||
groups = await getJson('/api/collection-suggestions');
|
||||
} catch (err) {
|
||||
el.textContent = `Couldn’t load suggestions: ${err.message}`;
|
||||
return;
|
||||
}
|
||||
if (!document.getElementById('aiSuggestions')) return;
|
||||
el.className = groups.length ? 'suggestion-list' : 'form-hint';
|
||||
el.innerHTML = groups.length
|
||||
? groups.map((g) => {
|
||||
const n = plural(g.models.length, 'model');
|
||||
return `
|
||||
<div class="suggestion" data-suggestion="${escapeHtml(g.name)}">
|
||||
<div class="suggestion-head">
|
||||
<strong>${escapeHtml(g.name)}</strong>
|
||||
<span class="form-hint">${g.project_id ? 'your collection' : 'new collection'} · ${n}</span>
|
||||
<span class="suggestion-actions">
|
||||
<button type="button" class="btn btn-primary btn-sm" data-ai="accept-suggestion">${g.project_id ? `Add ${n}` : `Create & add ${n}`}</button>
|
||||
<button type="button" class="btn btn-secondary btn-sm" data-ai="dismiss-suggestion">Dismiss</button>
|
||||
</span>
|
||||
</div>
|
||||
<div class="suggestion-models">${g.models.map((m) => `<button type="button" class="link-btn" data-open-model="${m.id}">${escapeHtml(m.name)}</button>`).join(', ')}</div>
|
||||
</div>`;
|
||||
}).join('')
|
||||
: 'No suggestions waiting. Tick “Suggest collections” when you start a run.';
|
||||
}
|
||||
|
||||
async function updateAiEstimate() {
|
||||
const el = document.getElementById('aiEstimate');
|
||||
const choice = fileGrid.querySelector('input[name="aiScope"]:checked');
|
||||
|
|
@ -1577,13 +1622,20 @@ async function handleAiStart(e) {
|
|||
e.preventDefault();
|
||||
const scope = fileGrid.querySelector('input[name="aiScope"]:checked').value;
|
||||
const maxCost = document.getElementById('aiMaxCost').value;
|
||||
const checked = (id) => document.getElementById(id).checked;
|
||||
if (!checked('aiDescribe') && !checked('aiRename') && !checked('aiCollections')) {
|
||||
alert('Choose at least one thing for the AI to do.');
|
||||
return;
|
||||
}
|
||||
const btn = document.getElementById('aiStartBtn');
|
||||
btn.disabled = true;
|
||||
try {
|
||||
await postJson('/api/ai/runs', {
|
||||
scope,
|
||||
overwriteDescriptions: document.getElementById('aiOverwrite').checked,
|
||||
renameUnclear: document.getElementById('aiRename').checked,
|
||||
describe: checked('aiDescribe'),
|
||||
overwriteDescriptions: checked('aiDescribe') && checked('aiOverwrite'),
|
||||
renameUnclear: checked('aiRename'),
|
||||
suggestCollections: checked('aiCollections'),
|
||||
maxCost: maxCost ? Number(maxCost) : null,
|
||||
});
|
||||
aiState.expectChange = true;
|
||||
|
|
@ -1618,6 +1670,19 @@ async function handleAiPanelClick(e) {
|
|||
await postJson(`/api/ai/runs/${id}/resume`);
|
||||
await refreshAiPanel();
|
||||
break;
|
||||
case 'accept-suggestion': {
|
||||
btn.disabled = true;
|
||||
const result = await postJson('/api/collection-suggestions/accept', { name: btn.closest('[data-suggestion]').dataset.suggestion });
|
||||
await loadData(); // the sidebar gains the collection
|
||||
renderCollectionSuggestions();
|
||||
resultCount.textContent = `Added ${plural(result.added, 'model')} to ${result.project.name}`;
|
||||
break;
|
||||
}
|
||||
case 'dismiss-suggestion':
|
||||
btn.disabled = true;
|
||||
await postJson('/api/collection-suggestions/dismiss', { name: btn.closest('[data-suggestion]').dataset.suggestion });
|
||||
renderCollectionSuggestions();
|
||||
break;
|
||||
case 'delete-rule':
|
||||
aiState.tagRules = await getJson(`/api/autotag/rules/${id}`, { method: 'DELETE' });
|
||||
renderRulesCard();
|
||||
|
|
|
|||
|
|
@ -436,3 +436,14 @@ label { display: block; font-size: 13px; font-weight: 500; color: var(--text-mut
|
|||
@media (max-width: 640px) { .ai-grid { grid-template-columns: 1fr; } .ai-run-actions { margin-left: 0; } }
|
||||
.renamed-note { font-size: 13px; color: var(--text); background: var(--accent-soft); border: 1px solid rgba(240, 151, 42, 0.35); border-radius: var(--radius-sm); padding: 8px 12px; margin-bottom: 16px; }
|
||||
.renamed-from { font-size: 12px; color: var(--text-dim); }
|
||||
.ai-tasks { border: none; padding: 0; margin: 0; display: flex; flex-direction: column; gap: 8px; }
|
||||
.ai-tasks legend { font-size: 11px; font-weight: 600; text-transform: uppercase; letter-spacing: 0.08em; color: var(--text-dim); margin-bottom: 8px; }
|
||||
.ai-start .ai-subtask { margin-left: 26px; }
|
||||
.ai-start label:has(input:disabled) { opacity: 0.5; cursor: default; }
|
||||
.suggestion-list { display: flex; flex-direction: column; gap: 10px; }
|
||||
.suggestion { border: 1px solid var(--border); border-radius: var(--radius-sm); padding: 10px 14px; background: var(--surface-2); display: flex; flex-direction: column; gap: 6px; }
|
||||
.suggestion-head { display: flex; flex-wrap: wrap; align-items: center; gap: 6px 12px; font-size: 14px; }
|
||||
.suggestion-actions { margin-left: auto; display: flex; gap: 8px; }
|
||||
.suggestion-actions .btn-sm { align-self: auto; }
|
||||
.suggestion-models { font-size: 13px; color: var(--text-muted); line-height: 1.7; }
|
||||
.suggested-collections { font-size: 12px; color: var(--accent); }
|
||||
|
|
|
|||
34
server.js
34
server.js
|
|
@ -1208,8 +1208,9 @@ app.get('/api/ai/runs', async (req, res, next) => {
|
|||
|
||||
app.post('/api/ai/runs', async (req, res, next) => {
|
||||
try {
|
||||
const { scope, ids, overwriteDescriptions, renameUnclear, maxCost } = req.body || {};
|
||||
const run = await aiRuns.createRun(await aiScopeIds(scope, ids), { overwriteDescriptions, renameUnclear, maxCost });
|
||||
const { scope, ids, describe, overwriteDescriptions, renameUnclear, suggestCollections, maxCost } = req.body || {};
|
||||
const run = await aiRuns.createRun(await aiScopeIds(scope, ids),
|
||||
{ describe, overwriteDescriptions, renameUnclear, suggestCollections, maxCost });
|
||||
res.status(201).json(run);
|
||||
} catch (e) { next(e); }
|
||||
});
|
||||
|
|
@ -1236,6 +1237,35 @@ app.post('/api/ai/runs/:id/resume', async (req, res, next) => {
|
|||
} catch (e) { next(e); }
|
||||
});
|
||||
|
||||
// ----------------------------------------------------------------- collection suggestions
|
||||
function suggestionTarget(body) {
|
||||
const name = typeof (body && body.name) === 'string' ? body.name.trim() : '';
|
||||
const modelIds = Array.isArray(body && body.modelIds) ? body.modelIds.map(Number).filter(Number.isInteger) : null;
|
||||
return { name, modelIds: modelIds && modelIds.length ? modelIds : null };
|
||||
}
|
||||
|
||||
app.get('/api/collection-suggestions', async (req, res, next) => {
|
||||
try { res.json(await store.listCollectionSuggestions()); } catch (e) { next(e); }
|
||||
});
|
||||
|
||||
app.post('/api/collection-suggestions/accept', async (req, res, next) => {
|
||||
try {
|
||||
const { name, modelIds } = suggestionTarget(req.body);
|
||||
if (!name) return res.status(400).json({ error: 'Which suggestion?' });
|
||||
const result = await store.acceptCollectionSuggestion(name, modelIds);
|
||||
if (!result) return res.status(404).json({ error: 'That suggestion is no longer waiting' });
|
||||
res.json(result);
|
||||
} catch (e) { next(e); }
|
||||
});
|
||||
|
||||
app.post('/api/collection-suggestions/dismiss', async (req, res, next) => {
|
||||
try {
|
||||
const { name, modelIds } = suggestionTarget(req.body);
|
||||
if (!name) return res.status(400).json({ error: 'Which suggestion?' });
|
||||
res.json({ dismissed: await store.dismissCollectionSuggestion(name, modelIds) });
|
||||
} catch (e) { next(e); }
|
||||
});
|
||||
|
||||
// ----------------------------------------------------------------- semantic search
|
||||
app.get('/api/search', async (req, res, next) => {
|
||||
try {
|
||||
|
|
|
|||
|
|
@ -107,6 +107,45 @@ test('a model that refuses JSON mode and images is retried without them', async
|
|||
}
|
||||
});
|
||||
|
||||
test('collection ideas are asked for with the library’s collections, and tidied', () => {
|
||||
const [system, user] = ai.buildMessages(MODEL, { describe: false, suggestCollections: true, collectionNames: ['Bookends', 'Halloween'] });
|
||||
assert.match(system.content, /"collections"/);
|
||||
assert.doesNotMatch(system.content, /"tags"/);
|
||||
assert.match(user.content, /Existing collections in this library: Bookends, Halloween/);
|
||||
assert.match(ai.buildMessages(MODEL, { suggestCollections: true })[1].content, /no collections yet/);
|
||||
|
||||
const reply = '{"collections": ["bookends", " Halloween Decor ", "Book-ends", "Dragons", "Toys"]}';
|
||||
const r = ai.parseReply(reply, { describe: false, collections: true });
|
||||
assert.deepEqual(r.collections, ['bookends', 'Halloween Decor', 'Dragons']);
|
||||
assert.deepEqual(r.tags, []); // not asked for, so an answer without tags is fine
|
||||
assert.deepEqual(ai.parseReply('{"tags": ["dragon"], "collections": ["Dragons"]}').collections, []);
|
||||
});
|
||||
|
||||
test('a reply that isn’t JSON is asked again once, and both answers count towards cost', async () => {
|
||||
const provider = await fakeProvider([
|
||||
() => reply('Sure! Tags: dragon, toy'),
|
||||
() => reply('{"tags": ["dragon"], "description": "A dragon."}'),
|
||||
]);
|
||||
try {
|
||||
const result = await ai.describeModel(provider.config, MODEL);
|
||||
assert.deepEqual(result.tags, ['dragon']);
|
||||
assert.equal(result.replyRetried, true);
|
||||
assert.deepEqual(result.usage, { prompt: 2000, completion: 200 });
|
||||
assert.ok(Math.abs(result.cost - 0.00084) < 1e-9);
|
||||
assert.equal(provider.requests.length, 2);
|
||||
} finally {
|
||||
await provider.close();
|
||||
}
|
||||
|
||||
const stubborn = await fakeProvider([() => reply('no JSON here')]);
|
||||
try {
|
||||
await assert.rejects(ai.describeModel(stubborn.config, MODEL), /JSON/);
|
||||
assert.equal(stubborn.requests.length, 2);
|
||||
} finally {
|
||||
await stubborn.close();
|
||||
}
|
||||
});
|
||||
|
||||
test('a rejected key is fatal for a run; a bad reply is not', async () => {
|
||||
const provider = await fakeProvider([() => [401, { error: { message: 'Authentication Fails' } }]]);
|
||||
try {
|
||||
|
|
|
|||
Loading…
Reference in a new issue