Skip to content

Commit ca70d6b

Browse files
MariomarqueztMariomarquezt
authored
feat(ai): add OpenAI-compatible provider (custom base URL)
Adds a generic OpenAI-compatible provider, shares the chat/completions adapter with Ollama, and fixes provider credential testing so empty live model catalogues are reported as failed tests.\n\nVerified with bun test, bun run build, and bun run lint.
1 parent 1b924f2 commit ca70d6b

16 files changed

Lines changed: 757 additions & 384 deletions

File tree

‎docs/features/agent.md‎

Lines changed: 31 additions & 13 deletions
Original file line numberDiff line numberDiff line change
@@ -2,7 +2,7 @@
22

33
The AI Agent is a model-powered assistant integrated into the visual editor. The user types a request in the Agent Panel; the agent reads the current page snapshot, plans a sequence of edits, and executes them by calling tools. Structure is written as semantic HTML (`insertHtml` / `replaceNodeHtml`); styling is written as CSS — a `<style>` block and/or `class=` attributes inside the insert, or the dedicated `applyCss` tool for authoring/editing any CSS on its own. There is one CSS path and it accepts every selector; `assignClass` / `removeClass` attach existing classes to nodes.
44

5-
The agent runs on a provider-agnostic AI runtime (`server/ai/`) that can drive any supported model (Anthropic Claude, OpenAI, OpenRouter, Ollama). Every driver talks directly to its provider's REST API over HTTP/SSE — no provider SDKs. All four share one multi-turn tool loop (`drivers/http/toolLoop.ts`); each supplies only a small `ProviderAdapter` of pure mapping functions. The plain `@anthropic-ai/sdk` (and any provider SDK) is banned repo-wide. Gated by `ai-driver-isolation.test.ts`.
5+
The agent runs on a provider-agnostic AI runtime (`server/ai/`) that can drive any supported model (Anthropic Claude, OpenAI, OpenRouter, Ollama, or any OpenAI-compatible endpoint). Every driver talks directly to its provider's REST API over HTTP/SSE — no provider SDKs. All drivers share one multi-turn tool loop (`drivers/http/toolLoop.ts`); each supplies only a small `ProviderAdapter` of pure mapping functions. The plain `@anthropic-ai/sdk` (and any provider SDK) is banned repo-wide. Gated by `ai-driver-isolation.test.ts`.
66

77
---
88

@@ -12,7 +12,7 @@ The agent runs on a provider-agnostic AI runtime (`server/ai/`) that can drive a
1212
- **Styling via CSS.** The agent emits CSS the same way a human pastes it: a `<style>` block and/or `class=` attributes inside the `insertHtml`/`replaceNodeHtml` payload, or the standalone `applyCss` tool. The importer (`cssToStyleRules`) classifies every selector — a bare `.foo {}` rule becomes a reusable Selectors-panel class bound to `class="foo"`; any other selector (`.hero a`, `a:hover`, `nav > li`) becomes an ambient rule; `style=` attributes land on the node's inline styles. There is no structured `classes` parameter — the agent never hand-builds classes node-by-node at insert time. `applyCss` is the single tool for authoring/editing CSS on its own; it **upserts**, so re-applying a selector edits the existing rule (the way descendant/pseudo rules get restyled).
1313
- **35 tools total.** 6 server-side catalog read tools (resolved server-side from the posted snapshot / DB) + 29 browser-bridged tools.
1414
- **Two-endpoint bridge.** `POST /admin/api/ai/chat/site` opens an NDJSON stream. When the model calls a browser-bridged tool, the server emits `toolRequest`; the browser executor reads or mutates the editor store and POSTs the `AiToolOutput` result to `POST /admin/api/ai/tool-result`.
15-
- **Provider-agnostic.** The runtime selects a driver (Anthropic, OpenAI, OpenRouter, Ollama) from the conversation's configured credential.
15+
- **Provider-agnostic.** The runtime selects a driver (Anthropic, OpenAI, OpenRouter, Ollama, Custom Provider) from the conversation's configured credential.
1616
- **Tool input schemas are a single source of truth** in `@core/ai` (`src/core/ai/toolSchemas.ts`). The server tool registry (`server/ai/tools/site/writeTools.ts`) and the browser executor (`executor.ts` + `tokenRunners.ts`) import the exact same schema objects — a constraint added once is enforced on both sides at build time. Gated by `ai-tool-schema-ssot.test.ts` and `ai-tools-typebox-only.test.ts`.
1717
- **Capabilities.** `ai.chat` required to stream; `ai.tools.write` required for write tools. Gated by `ai-handlers-capability-gated.test.ts`.
1818

@@ -56,16 +56,18 @@ server/ai/
5656
│ └── content/ — content-workspace tools (separate scope)
5757
├── drivers/
5858
│ ├── http/
59-
│ │ ├── sse.ts — parseSseStream(res): reassemble SSE frames across chunks
60-
│ │ ├── execTool.ts — executeAiTool(): server-handler vs browser-bridge dispatch; normaliseToolOutput(): wraps raw handler results in the canonical AiToolOutput envelope, validated via TypeBox (not duck-typed)
61-
│ │ ├── toolLoop.ts — runToolLoop(): provider-agnostic multi-turn loop
62-
│ │ ├── toolArgs.ts — parseToolArguments(json): shared tool-argument JSON parsing (one copy for all drivers)
63-
│ │ └── errors.ts — isAbortError / classifyHttpError
64-
│ ├── responses-shared.ts — OpenAI-Responses mapping + SSE translator + adapter factory (openai + openrouter)
65-
│ ├── anthropic.ts — Anthropic driver: direct POST /v1/messages (no SDK)
66-
│ ├── openai.ts — OpenAI driver: direct POST /v1/responses (no SDK)
67-
│ ├── openrouter.ts — OpenRouter driver: direct POST /v1/responses (shared Responses path; live /models; native cost)
68-
│ └── ollama.ts — Ollama driver: direct POST /v1/chat/completions (no SDK)
59+
│ │ ├── sse.ts — parseSseStream(res): reassemble SSE frames across chunks
60+
│ │ ├── execTool.ts — executeAiTool(): server-handler vs browser-bridge dispatch; normaliseToolOutput(): wraps raw handler results in the canonical AiToolOutput envelope, validated via TypeBox (not duck-typed)
61+
│ │ ├── toolLoop.ts — runToolLoop(): provider-agnostic multi-turn loop
62+
│ │ ├── toolArgs.ts — parseToolArguments(json): shared tool-argument JSON parsing (one copy for all drivers)
63+
│ │ ├── chatCompletions.ts — shared /v1/chat/completions SSE adapter (makeChatCompletionsAdapter); used by ollama + openai-compatible
64+
│ │ └── errors.ts — isAbortError / classifyHttpError
65+
│ ├── responses-shared.ts — OpenAI-Responses mapping + SSE translator + adapter factory (openai + openrouter)
66+
│ ├── anthropic.ts — Anthropic driver: direct POST /v1/messages (no SDK)
67+
│ ├── openai.ts — OpenAI driver: direct POST /v1/responses (no SDK)
68+
│ ├── openrouter.ts — OpenRouter driver: direct POST /v1/responses (shared Responses path; live /models; native cost)
69+
│ ├── ollama.ts — Ollama driver: POST /v1/chat/completions via shared chatCompletions adapter; live /api/tags catalogue
70+
│ └── openaiCompatible.ts — Custom Provider driver: any /v1/chat/completions endpoint; live GET /v1/models catalogue
6971
└── runtime/
7072
├── runner.ts — runChat(): drives a driver, emits stream events
7173
├── persister.ts — ConversationsPersister: messages + usage to DB; writes contextTokens snapshot
@@ -129,6 +131,22 @@ The composer area includes a `<ContextMeter>` that shows "context used / window"
129131

130132
---
131133

134+
## Providers
135+
136+
Each entry in **Settings → AI → Providers** stores one credential. The provider id is fixed; the auth mode and input fields are derived from it — the UI never asks you to choose.
137+
138+
| Provider | Label in UI | Auth mode | Required field | Optional field | Model discovery |
139+
|---|---|---|---|---|---|
140+
| `anthropic` | Anthropic (Claude) | `apiKey` | API key (`sk-ant-…`) | — | Static `claude-*` catalogue enriched with OpenRouter prices + context windows |
141+
| `openai` | OpenAI | `apiKey` | API key (`sk-…`) | — | Static `gpt-*` / `o*` catalogue enriched with OpenRouter prices + context windows |
142+
| `openrouter` | OpenRouter | `apiKey` | API key (`sk-or-…`) | — | Live `GET /api/v1/models` (cross-provider; native cost reporting) |
143+
| `ollama` | Ollama (local) | `baseUrl` | Base URL (e.g. `http://localhost:11434`) | API key (bearer, for proxied deployments) | Live `GET {baseUrl}/api/tags`; static fallback list when unreachable |
144+
| `openai-compatible` | Custom Provider | `baseUrl` | Base URL — any host serving the OpenAI `/v1/chat/completions` wire protocol | API key (bearer; cloud services need one, local servers often don't) | Live `GET {baseUrl}/v1/models` (standard OpenAI list shape); model `id` used as label |
145+
146+
**Custom Provider** (id `openai-compatible`) is the generic adapter for any endpoint that speaks the OpenAI chat/completions wire protocol — Groq (`https://api.groq.com/openai`), Together, DeepSeek, Mistral, Fireworks, self-hosted vLLM, LM Studio, and others. Capabilities default to `{ toolCalling: true, visionInput: false, promptCache: false, streaming: true }`; the operator is responsible for selecting a model that actually supports tool calling. Because arbitrary endpoints are not in the OpenRouter catalogue, no context-window enrichment is available and the context meter stays hidden for these models.
147+
148+
---
149+
132150
## Flow
133151

134152
```text
@@ -561,7 +579,7 @@ The `<ContextMeter>` shows how much of the active model's context window the cur
561579
- **Window** (`windowTokens` prop from `AgentPanel`): the model's max total tokens, resolved once from `GET /admin/api/ai/providers/:id/models?credentialId=…`. The models endpoint enriches Anthropic and OpenAI models with `contextWindow` from the live OpenRouter catalogue (`server/ai/pricing/`); OpenRouter populates it from its own native fetch. Ollama models and uncatalogued models have no window — the meter hides.
562580
- **Used** (`agentContextTokens` in the store): the provider-normalised "context used" — the CURRENT context size, computed by `normalizeContextTokens(providerId, buckets)` in `server/ai/contextTokens.ts`:
563581
- Anthropic reports `input_tokens` excluding cache buckets, so the true total is `promptTokens + cacheReadTokens + cacheCreationTokens`.
564-
- OpenAI / OpenRouter / Ollama report `input_tokens` as the full input; `promptTokens` alone is the total.
582+
- OpenAI / OpenRouter / Ollama / Custom Provider report `input_tokens` as the full input; `promptTokens` alone is the total.
565583

566584
**Live, per-round, not summed.** A turn makes one provider round-trip per tool batch. The toolLoop emits a `context` event **each round** carrying THAT round's input buckets; the chat handler injects the normalised `contextTokens` and the browser updates the meter on every round — so it climbs *during* a long tool loop instead of only at the end. The meter is the LATEST round's input (the current window fill), never the sum across rounds (which would over-count, since each round re-sends the growing context). The terminal `usage` event is **billing only** — its `promptTokens` stays summed across rounds (you pay input per round). The persister keeps the latest `context` value in memory (`recordContext`) and writes it once to `ai_conversations.context_tokens` with the final `usage` (overwritten per turn), so `loadAgentConversation` restores the true context on reload.
567585

‎server/ai/contextTokens.ts‎

Lines changed: 3 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -5,8 +5,9 @@
55
*
66
* - Anthropic reports `input_tokens` EXCLUDING the cache buckets, so the true
77
* total is prompt + cacheRead + cacheCreation.
8-
* - OpenAI / OpenRouter / Ollama report `input_tokens` as the full input (any
9-
* cached tokens are already a subset), so prompt alone is the total.
8+
* - OpenAI / OpenRouter / Ollama / Custom Provider report `input_tokens` as
9+
* the full input (any cached tokens are already a subset), so prompt alone
10+
* is the total.
1011
*
1112
* Two callers share this: the chat handler injects the value onto the wire
1213
* `usage` event for the live meter, and the persister writes it to the

‎server/ai/drivers/http/chatCompletions.test.ts‎

Lines changed: 96 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,96 @@
1+
import { describe, it, expect } from 'bun:test'
2+
import {
3+
mapChatHistory,
4+
ChatCompletionsTurnTranslator,
5+
trimSlash,
6+
normalizeOpenAiBaseUrl,
7+
} from './chatCompletions'
8+
import type { SseFrame } from './sse'
9+
10+
function frame(obj: unknown): SseFrame {
11+
return { event: null, data: JSON.stringify(obj) }
12+
}
13+
14+
describe('chatCompletions shared adapter', () => {
15+
it('trimSlash strips trailing slashes', () => {
16+
expect(trimSlash('http://x/v1/')).toBe('http://x/v1')
17+
expect(trimSlash('http://x/v1')).toBe('http://x/v1')
18+
})
19+
20+
it('normalizeOpenAiBaseUrl strips trailing /v1 so it is not doubled when building the endpoint', () => {
21+
// With /v1 suffix — should strip it so appending /v1/... is correct.
22+
expect(normalizeOpenAiBaseUrl('https://api.groq.com/openai/v1')).toBe('https://api.groq.com/openai')
23+
expect(normalizeOpenAiBaseUrl('https://api.groq.com/openai/v1/')).toBe('https://api.groq.com/openai')
24+
// Without /v1 suffix — no-op.
25+
expect(normalizeOpenAiBaseUrl('https://api.groq.com/openai')).toBe('https://api.groq.com/openai')
26+
// Ollama-style URL with no path — no-op.
27+
expect(normalizeOpenAiBaseUrl('http://localhost:11434')).toBe('http://localhost:11434')
28+
expect(normalizeOpenAiBaseUrl('http://localhost:11434/')).toBe('http://localhost:11434')
29+
})
30+
31+
it('mapChatHistory prepends the system prompt as a system message', () => {
32+
const turns = mapChatHistory(['be terse'], [
33+
{ role: 'user', content: [{ kind: 'text', text: 'hi' }] },
34+
])
35+
expect(turns[0]).toEqual([{ role: 'system', content: 'be terse' }])
36+
expect(turns[1]).toEqual([{ role: 'user', content: 'hi' }])
37+
})
38+
39+
it('translator accumulates streamed text and finishes with stop=true when no tool calls', () => {
40+
const t = new ChatCompletionsTurnTranslator()
41+
const events = t.translate(frame({ choices: [{ delta: { content: 'Hello' } }] }))
42+
expect(events).toEqual([{ type: 'text', text: 'Hello' }])
43+
const result = t.finish()
44+
expect(result.stop).toBe(true)
45+
expect(result.toolCalls).toEqual([])
46+
})
47+
48+
// Real OpenAI-compatible gateways (OpenCode Zen, OpenRouter, …) send explicit
49+
// `null` for optional per-chunk fields rather than omitting them. The chunk
50+
// schema must tolerate these, or `parseValue` throws, the frame is dropped,
51+
// and the model's entire reply silently vanishes ("no reply").
52+
it('still emits text when the chunk carries usage:null', () => {
53+
const t = new ChatCompletionsTurnTranslator()
54+
const events = t.translate(frame({ choices: [{ delta: { content: 'Hi' } }], usage: null }))
55+
expect(events).toEqual([{ type: 'text', text: 'Hi' }])
56+
})
57+
58+
it('still emits text when delta.tool_calls is null', () => {
59+
const t = new ChatCompletionsTurnTranslator()
60+
const events = t.translate(
61+
frame({ choices: [{ delta: { content: 'Hi', reasoning_content: null, tool_calls: null }, finish_reason: 'stop' }], usage: null }),
62+
)
63+
expect(events).toEqual([{ type: 'text', text: 'Hi' }])
64+
})
65+
66+
it('captures the final content of a reasoning model (content empty during reasoning, filled at the end)', () => {
67+
const t = new ChatCompletionsTurnTranslator()
68+
// Reasoning phase: content is "" (or null), answer lives in reasoning_content; tool_calls/usage are null.
69+
t.translate(frame({ choices: [{ delta: { content: '', reasoning_content: 'thinking…', tool_calls: null } }], usage: null }))
70+
t.translate(frame({ choices: [{ delta: { content: null, reasoning_content: ' more' } }], usage: null }))
71+
// Final answer arrives in content.
72+
const last = t.translate(
73+
frame({ choices: [{ delta: { content: 'Hello there!', tool_calls: null }, finish_reason: 'stop' }], usage: null }),
74+
)
75+
expect(last).toEqual([{ type: 'text', text: 'Hello there!' }])
76+
const result = t.finish()
77+
expect(result.stop).toBe(true)
78+
expect(result.assistantMessage?.[0]).toMatchObject({ role: 'assistant', content: 'Hello there!' })
79+
})
80+
81+
it('translator emits one toolCall event per accumulated call at finish_reason', () => {
82+
const t = new ChatCompletionsTurnTranslator()
83+
t.translate(frame({ choices: [{ delta: { tool_calls: [
84+
{ index: 0, id: 'c1', function: { name: 'insertHtml', arguments: '{"ht' } },
85+
] } }] }))
86+
const ev = t.translate(frame({ choices: [{ delta: { tool_calls: [
87+
{ index: 0, function: { arguments: 'ml":"<p/>"}' } },
88+
] }, finish_reason: 'tool_calls' }] }))
89+
const toolEvent = ev.find((e) => e.type === 'toolCall')
90+
expect(toolEvent).toBeTruthy()
91+
expect(toolEvent).toMatchObject({ type: 'toolCall', toolName: 'insertHtml', toolCallId: 'c1' })
92+
const result = t.finish()
93+
expect(result.stop).toBe(false)
94+
expect(result.toolCalls[0]).toMatchObject({ name: 'insertHtml' })
95+
})
96+
})

0 commit comments

Comments
 (0)