Skip to content

feat: agent web-fetch capability — new pluggable repo (ceilf6) wrapped as @frontagent/mcp-* (learned from CLAUDE-FABLE-5 web_fetch) #357

@ceilf6

Description

@ceilf6

Source learning (CLAUDE-FABLE-5.md)

CLAUDE-FABLE-5.md devotes large sections to web access toolingweb_fetch ("Fetch the contents of a web page at a given URL") and web_search, plus extensive when-to-search heuristics. This is one of the most prominent capabilities in the Fable-5 prompt that FrontAgent currently lacks.

Verified gaps (SCAN iteration 16):

  • grep -rniE "web_search|web_fetch|fetchUrl|searchWeb" packages/*/src apps/*/srcno matches.
  • @frontagent/mcp-web is playwright/browser-only (navigate/click/screenshot/get_page_structure/get_accessibility_tree/evaluate/close_browser); it has no plain HTTP fetch or HTML→text extraction.
  • (The CLAUDE-FABLE-5 skills capability is intentionally not proposed — FrontAgent already has a full skills system in packages/core/src/{skills,skill-content,skill-lab} (~3.7k LOC). No duplication.)

Why this is valuable for FrontAgent

FrontAgent is a frontend-engineering agent. During planning/execution it frequently needs current, authoritative reference material — framework/library API docs, migration guides, error-message resolutions, package READMEs. Today it can only reach the web by driving a full Playwright browser (heavy, stateful, slow, and aimed at testing the app under development, not at reading docs). A lightweight web_fetch (URL → cleaned, readable text/markdown) is the right tool for reference lookup and fits FrontAgent's "controlled perception" model when paired with URL allow/deny safety (cf. existing packages/mcp-web/src/url-safety.ts).

Coupling judgment → new pluggable repo under ceilf6

A URL→text fetcher is self-contained and low-coupling (no dependency on FrontAgent internals), exactly like ceilf6/filesense. Per the maintainer's integration rule (pluggable → separate ceilf6 repo; tightly-coupled → in-tree), this should be:

  1. A new standalone public repo under ceilf6 (proposed name websense — "agent-friendly web content sensing"; name open to maintainer), structured like Filesense-app: a small TypeScript library/CLI (fetch <url> → cleaned text/markdown + metadata), JSON output schema, URL-safety guards (block private/loopback/link-local hosts — SSRF protection), size/timeout limits, unit tests, MIT, engines.node >=20.
  2. Wrapped into FrontAgent as a new workspace package @frontagent/mcp-web-fetch (or @frontagent/mcp-websense), mirroring @frontagent/mcp-filesense (engine + tools.ts MCP adapter + tools.security.test.ts), then wired into the agent's MCP client registries (packages/runtime-node/src/mcp-clients.ts, apps/cli/src/mcp-client.ts).

Scope (this issue)

  • In scope: web_fetch only — fetch one URL, return cleaned readable text (HTML→markdown/text), with content-type handling, byte/time limits, redirect handling, and URL-safety (SSRF) guards. The standalone repo + the @frontagent/mcp-* adapter + registry wiring + minimal planner-prompt note that the tool exists.
  • Out of scope (deferred follow-on): web_search (requires a third-party search provider + API key/config — its own later iteration), and the full CLAUDE-FABLE-5 when-to-search heuristic injection into the planner prompt (a coupled prompt-side follow-up once a search tool lands).

Acceptance criteria

  • New ceilf6/<name> public repo exists with: library/CLI fetch entrypoint, JSON output schema, URL-safety/SSRF guards with tests, size/timeout limits, README, MIT license, CI.
  • New @frontagent/mcp-web-fetch workspace package wraps it with an MCP tool definition + a tools.security.test.ts (asserts private/loopback/link-local URLs are rejected), following the @frontagent/mcp-filesense shape.
  • Tool registered in runtime-node + cli MCP client registries; pnpm quality:precommit green (lint + typecheck + test + test:workflows); Contract Guard / CI matrix green.
  • No regression to existing 96 desktop tests or package suites.

Workflow / guardrails

Follow the OSS Harness + repo-evolver flow: GitNexus impact analysis before editing integration points (mcp-clients.ts, cli/mcp-client.ts), detect_changes before commit, PR → develop with the GitNexus Impact Summary filled (this touches critical assembly surfaces in runtime-node), repo-guard review, CI matrix. The new standalone repo follows its own minimal CI like ceilf6/filesense.

Priority: P2 (genuine high-value capability gap learned from CLAUDE-FABLE-5; first item of the new evolutionary direction).

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type
    No fields configured for issues without a type.

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions