Skip to content

Build the agent-eval UI run routes #433

Open

Description

@joshblack

Part of #428.

This issue tracks building /runs and /runs/:id so we can browse saved benchmark and experiment runs and interpret their results over time.

Scope

  • Define and wire the result input for both agent-eval ui and agent-eval ui build, including how multiple result bundles are discovered. Do not hard-code this repository's results directory layout.
  • Provide a shared run/trial index and stable, collision-safe route IDs. Distinguish benchmark and experiment runs, multiple runs of the same resource, and runs created on the same date.
  • Read supported historical result schemas through the compatibility layer from Define way for result artifacts to be durable across schema updates #316, normalizing them for the UI rather than introducing page-specific schema handling.
  • List runs with their source benchmark/experiment and available creation metadata, with deterministic ordering and a bounded browsing strategy for growing history.
  • Show a run's saved configuration, result summary, model/treatment/scenario breakdown, and links to individual trials and available source definitions.
  • Keep historical runs viewable when current definitions are renamed, changed, or removed. Clearly distinguish saved metadata from current configuration.
  • Handle no results, unknown IDs, unsupported schema versions, malformed bundles, and missing trial files explicitly. One unsupported bundle should not silently erase history or hide unrelated supported runs.
  • Keep expensive trial/artifact details out of the initial collection payload and load them when needed.
  • Use Primer components and design tokens, support light/dark mode, and provide accessible navigation.
  • Verify local preview and static export, including direct links, deployment base paths, mixed supported schema versions, repeated same-day runs, and results without current source definitions.

Dependencies

Blocked by #316 for supported historical result ingestion. Layout and current-schema prototyping can proceed while that foundation is developed. This issue owns the shared result discovery/identity contract consumed by the homepage and other resource pages; the trials issue depends on that contract, not on completing every run-page UI detail.

Starting points

  • packages/agent-eval/src/cli/commands/ui.ts:11-15,39-53 currently accepts configuration directories but has no result input.
  • website/src/runs.ts:8-17,132 uses a repository-specific experiment-results directory and a date as the run ID. This needs generalization for a global run route.
  • website/src/result-files.ts:60-77 currently skips invalid/incompatible bundles. Define way for result artifacts to be durable across schema updates #316 needs to define the supported compatibility boundary and diagnostics.

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions