AX-300 · Lesson 10 of 12 · Module 4: Ship and Improve · 3:14
Watch every run
Every run leaves a trace, the step-by-step record of its plan, each tool call with its inputs and result, each decision and each approval; when something goes wrong, open the trace first. A run that finishes can still hide a failure, so watch a few health signals each week (runs finished, stopped to ask, failed tool calls, declined approvals, cost per run) and set alerts for the dangerous ones (a tool failing twice, a run past its limits, a never-list attempt, a failed guardrail check). Keep audit logs of every action and approval, who, what and when, with a set retention and no customers’ details, and turn each new failure into an eval case. Worked example: a failing stock tool and an out-of-date note led to a guessed “back by Friday”; the failed-calls signal had jumped from 1 to 14.

What you will learn
- Read a trace to find why a run went wrong
- Choose the signals and alerts to watch
- Keep logs useful without personal data
The one thing to remember
Watch the signals, and read the traces.
Common mistake: Watching only whether runs finish
A run can finish on a guess after a tool fails, and look exactly like a success.
The fix: Watch tool failures and stops too, set alerts, and open traces every week.
Key terms
- Trace
- The step-by-step record of one agent run.
- Health signal
- A number you watch each week, like failed tool calls.
- Audit log
- A lasting record of who did what, and when.
Part of AI Agent Builder (AX-300)
For people who run AI workflows: build an agent with the right tools, tests and guardrails, and approvals where it counts.
12 lessons, practice quizzes, a final exam and the Sophyx Certified AI Agent Builder – Professional certificate. Regular price USD 49.99. Your first certification is free. Free with any paid Sophyx plan.
Start the course free