Skip to main content
Sofia works best as an iterative collaborator. Start with the outcome you want, let it investigate the relevant Coval data, and then refine or act on what it finds.

​
Write an effective request

A useful request usually contains three things:
  1. Goal: what decision or outcome you need.
  2. Scope: which agent, run, conversations, metric, test set, or time range Sofia should inspect.
  3. Output: how you want the result organized and whether Sofia should only investigate or also propose changes.
For example:
Analyze the last seven days of conversations for the support agent. Group the most common resolution failures and recommend test case additions to cover each failure mode. Do not make changes yet.

​
Common workflows

​
Work from the current page

Sofia receives context about the page and resource you are viewing. Start from a focused page when possible:
  • From Sofia AI, ask about a finding from Sofia’s scheduled investigations, what it affects, and what to change first.
  • From a run or simulation, ask why cases failed, what changed, or which example deserves attention first.
  • From the Uploaded Conversations page, ask for recurring patterns, customer pain, operational issues, or metric ideas.
  • From a test set, ask for missing scenarios, duplicates, or low-value cases.
  • From a metric, ask how to make the evaluation more precise or how its outcomes compare with Human Review.
  • From a dashboard or report, ask what moved, what should be investigated, or what should be tracked next.
  • From an alert, ask why it fired, whether the threshold is right, or which conversations triggered it.
If the page summary emphasizes the wrong thing, tell Sofia what you care about. For example: “Prioritize metric regressions for Tool Call success.”

​
Ask for evidence and scope

When the result will drive a product decision, ask Sofia to make the investigation auditable:
  • “How many conversations did you analyze?”
  • “What time range and filters did you use?”
  • “Link the examples supporting each conclusion.”
  • “Separate confirmed findings from hypotheses.”
  • “What data was unavailable or excluded?”
Sofia can navigate to supported resources and link simulations, conversations, runs, and metrics from its answers. Open those links to verify the source data or continue the investigation in the product. For simulation or uploaded-conversation volume, specify a date range and time zone. Counts cover non-deleted records from all creators in the current workspace, including failed and in-progress records; they are not attributed to an individual user. Recent activity may take time to appear. Ask Sofia to label partial days or weeks when comparing averages, and narrow the date range if a read cannot complete.

​
Review actions before they run

Every change Sofia proposes appears in a card beneath its reply, and nothing is written until you act on the card. Only two kinds of action run on their own: taking you to a page, and rendering a chart in the conversation. Everything else waits for you.
  • Confirm or dismiss. Each card names the action and the resource it targets. Select Confirm to run it or Dismiss to drop it.
  • Edits show what changes. When Sofia proposes updating an existing resource, the card explains the change and lists the new values under What changes, so you can compare them with what is there today.
  • Deletions take two steps. Cards for deletions are styled as destructive and state the consequence. Select Delete, then select the named delete button that replaces it to run the deletion. Deleting a test set also asks you to type the test set’s exact name first.
  • Expired proposals. A card that sits unconfirmed for too long can no longer be run. Select Propose again and Sofia produces a fresh proposal for you to review.
When you have a test set open in its editor, Sofia stages its test case changes in that editor instead of writing them directly. Review the staged changes, then Save the test set to apply them or Undo the ones you do not want.
Read the confirmation card carefully. Deletions and edits can affect shared evaluation resources used by other members of your organization.
After an action completes, use its receipt or destination link to verify the result. If an action fails, ask Sofia to explain the error and propose a safe next step rather than repeating it without review.

​
Know when to narrow the task

Sofia may need a smaller scope when a request spans a large number of conversations or several different goals. Split the work when:
  • you need detailed evidence across a long time range;
  • the analysis mixes setup, diagnosis, and resource creation;
  • you need a precise comparison across several agents or versions;
  • Sofia reports that only part of the requested data was analyzed.
A good pattern is to ask for a high-level census first, select the most important segment, and then request a deeper analysis with examples.

​
Share feedback

Use the thumbs-up or thumbs-down control on a Sofia response to tell the Coval team whether it was useful. When something is wrong, follow with a short correction that identifies the missing data, incorrect conclusion, or action you expected. This helps improve both the current conversation and future Sofia behavior.