This directory hosts the reusable task scenarios that power MCP evaluations for the Kubernetes MCP Server. Each task captures a realistic cluster workflow (setup, agent-driven actions, verification, and cleanup) so different agents can be compared against the same benchmark.
- Core tasks – core cluster workflows such as creating pods, fixing deployments, managing RBAC, or debugging state issues.
- Config tasks – workflows that exercise the configuration toolset (contexts, current config).
- Helm tasks – workflows that exercise the Helm toolset (install, list, uninstall releases).
- Kiali tasks – service-mesh and observability workflows that exercise the Kiali MCP toolset (Istio config, topology, mesh health, tracing).
- KubeVirt tasks – virtual machine management workflows that exercise the KubeVirt MCP toolset (VM creation, lifecycle management, resource updates).
- Tekton tasks – CI/CD workflows that exercise the Tekton toolset (pipelines, tasks, pipeline runs).
- NetObserv tasks – network flow and metrics workflows that exercise the NetObserv MCP toolset (flows, metrics, export).
Most subdirectories under core/, config/, helm/, kiali/, kubevirt/, netobserv/, or tekton/ define a single scenario (a few, like kiali/scripts/, kubevirt/helpers/, and tekton/helpers/, hold shared helpers instead):
*.yaml– declarative description consumed by the evaluation harness (prompts, success criteria, required tools).setup.sh/verify.sh/cleanup.sh– shell hooks (optional) that prime the cluster, assert post-conditions, and reset resources so tasks stay idempotent.artifacts/– supporting manifests, scripts, or payloads referenced by the task definition.
- Pick the closest family and create a new subfolder.
- Author the task YAML referencing MCP tools, expected observations, and any artifacts.
- Provide helper scripts if the scenario needs deterministic setup or verification.
- Add project metadata so the task appears in the README validated-projects table (unlabeled tasks are skipped).
- The toolset column comes from the
requireslabel, notproject; setrequireswhen a scenario needs a non-core toolset. projectis a label;project-nameandproject-urlare annotations. The generator panics ifprojectis set without both.- Example: helm/install-chart/install-chart.yaml.
- The toolset column comes from the
- Document nuances in a local
README.mdso future contributors and eval authors can replay the scenario manually.
Well-scoped, deterministic tasks make it easier to compare agents and regressions over time, so keep inputs minimal, outputs explicit, and always clean up what you create.
When a new MCP toolset lands, keep its evaluations isolated by creating a sibling directory under tasks/ named after the toolset (tasks/<name>, etc.). Populate it with:
- A scoped
README.mddescribing the toolset focus and prerequisite context. - One subfolder per scenario that follows the same layout described above (
*.yaml, scripts,artifacts/). - Any shared fixtures the stack needs (place them in a
shared/subdirectory if multiple scenarios reuse them).
This structure keeps task stacks discoverable and lets eval harnesses target toolset-specific workflows without mixing concerns from the core Kubernetes or Kiali libraries.