Add support for agent-eval ui and agent-eval ui build #428

Activity

1. joshblack

   added the

   Feature

   issue type on Oct 20, 2026
2. joshblack

   added sub-issues on Oct 20, 2026

   • Build the agent-eval UI homepage route #429
   • Build the agent-eval UI benchmark routes #430
   • Build the agent-eval UI experiment routes #431
   • Build the agent-eval UI scenario routes #432
   • Build the agent-eval UI run routes #433
   • Build the agent-eval UI trial routes #434
   • Define way for result artifacts to be durable across schema updates #316
3. joshblack commented on Oct 20, 2026

   

   joshblack

   on Oct 20, 2026

   MemberAuthor

   More actions

   Scaffolded the route work as sub-issues and added #316 to this parent so historical result support is tracked alongside the UI.

   Area	Issue	Routes
   Homepage	#429	/
   Benchmarks	#430	/benchmarks, /benchmark/:id
   Experiments	#431	/experiments, /experiment/:id
   Scenarios	#432	/scenarios, /scenario/:id
   Runs	#433	/runs, /runs/:id
   Trials	#434	/trials, /trials/:id
   Historical result compatibility	#316	Shared result ingestion

   Dependencies

   • Build the agent-eval UI run routes #433 is blocked by Define way for result artifacts to be durable across schema updates #316 so run history can combine supported result schemas.
   • Build the agent-eval UI trial routes #434 is blocked by Build the agent-eval UI run routes #433 for shared result discovery and run/trial identity, and transitively by Define way for result artifacts to be durable across schema updates #316.
   • These are native GitHub dependencies, not just references in the issue bodies. Layouts and fixture-backed UI can proceed in parallel; trial integration depends on the shared contract, not every run-page UI detail.
   • The homepage and configuration pages do not have hard blockers on each other or the result pages. They should integrate related-run/trial links as those routes land.
   • Add support for getting benchmarks, experiments, and scenarios by name or id #336 and feat!: hash resource IDs and separate ID/name lookups #422 are related identity work, not additional hard blockers added here.

   Blind spots and remaining planning

   • Result discovery is part of the work, not just rendering. The current UI command accepts benchmark, experiment, and scenario directories but no result input (packages/agent-eval/src/cli/commands/ui.ts:11-15,39-53). Build the agent-eval UI run routes #433 owns defining the result input for both local preview and static builds, discovery across bundles, and a shared index that the other pages can reuse.
   • Global route IDs need more than dates or resource-local trial IDs. The existing website identifies an experiment run by its date inside an experiment-specific directory (website/src/runs.ts:8-17,108,132). That is not sufficient by itself for /runs/:id across resources or multiple runs on the same date. Build the agent-eval UI run routes #433/Build the agent-eval UI trial routes #434 include collision-safe identity and direct-link stability. Coordinate resource identity with Add support for getting benchmarks, experiments, and scenarios by name or id #336/feat!: hash resource IDs and separate ID/name lookups #422.
   • Schema compatibility needs an explicit support policy. Define way for result artifacts to be durable across schema updates #316 proposes versioned schemas but leaves the supported range/cutoff open. We still need to settle treatment of unversioned artifacts, migration/normalization, unknown future versions, and mixed valid/invalid bundles. The current reader skips invalid or incompatible bundles (website/src/result-files.ts:60-77); the UI should distinguish unsupported history from genuinely having no results. Historical views should use recorded configuration even when current definitions change or disappear.
   • The requested detail URLs differ from the experimental UI. Current links use /benchmarks/:id, /experiments/:id, and /scenarios/:id (packages/agent-eval/src/ui/app/page.tsx:29,41,53). Build the agent-eval UI benchmark routes #430-Build the agent-eval UI scenario routes #432 use the requested singular detail routes and include deciding whether to retain aliases. Any alias approach needs to work on a static host, not assume a redirect server.
   • Shared shell and release readiness are not separate issues yet. A common navigation/layout, consistent empty/error states, and final cross-route integration still need an owner. The packaged layout currently just renders the page children (packages/agent-eval/src/ui/app/layout.tsx:17-19), and the command remains feature-flagged (packages/agent-eval/src/cli.ts:15-20). Before calling Add support for agent-eval ui and agent-eval ui build #428 stable, we should also cover installed-package behavior outside this repository, command help/configuration, and local/static parity. These are remaining parent-level planning items, not extra issues created in this pass.
   • Static publication changes the artifact boundary. Build the agent-eval UI trial routes #434 includes lazy loading and bundle-contained artifact paths. We should also explicitly decide which transcripts, prompts, workspace files, and media are safe to publish, and how missing artifacts are represented. Growing history should not put every transcript/file preview into collection-page payloads.

   Each route issue includes local/static navigation, deployment base paths, accessible Primer UI, light/dark mode, and relevant missing-data cases. High-level homepage statistics remain a follow-up rather than blocking this scope. Retry/resume controls and live execution monitoring are not implied by browsing saved runs.
4. joshblack

   self-assigned this

   on Oct 22, 2026