AX-300 · Lesson 10 of 12 · Module 4: Ship and Improve · 3:14

Watch every run

Every run leaves a trace, the step-by-step record of its plan, each tool call with its inputs and result, each decision and each approval; when something goes wrong, open the trace first. A run that finishes can still hide a failure, so watch a few health signals each week (runs finished, stopped to ask, failed tool calls, declined approvals, cost per run) and set alerts for the dangerous ones (a tool failing twice, a run past its limits, a never-list attempt, a failed guardrail check). Keep audit logs of every action and approval, who, what and when, with a set retention and no customers’ details, and turn each new failure into an eval case. Worked example: a failing stock tool and an out-of-date note led to a guessed “back by Friday”; the failed-calls signal had jumped from 1 to 14.

Frame from lesson 10 of Sophyx Certified AI Agent Builder – Professional, "Watch every run"

What you will learn

  • Read a trace to find why a run went wrong
  • Choose the signals and alerts to watch
  • Keep logs useful without personal data

The one thing to remember

Watch the signals, and read the traces.

Common mistake: Watching only whether runs finish

A run can finish on a guess after a tool fails, and look exactly like a success.

The fix: Watch tool failures and stops too, set alerts, and open traces every week.

Key terms

Trace
The step-by-step record of one agent run.
Health signal
A number you watch each week, like failed tool calls.
Audit log
A lasting record of who did what, and when.

Part of AI Agent Builder (AX-300)

For people who run AI workflows: build an agent with the right tools, tests and guardrails, and approvals where it counts.

12 lessons, practice quizzes, a final exam and the Sophyx Certified AI Agent Builder – Professional certificate. Regular price USD 49.99. Your first certification is free. Free with any paid Sophyx plan.

Start the course free
Book a Demo