by AgentPulse (LX AI)
Agent observability is the practice of recording what an autonomous agent decided, which tools it called, and what came back — as a replayable sequence. Classic APM was built around a request that has a start, an end, and a status code. An agent run has none of those guarantees.
Key takeaways
Application Performance Monitoring earned its keep by answering questions about services: is the checkout API slow, is the database saturated, did error rates spike after a deploy. Those questions still matter for the API gateway in front of your agent. They do not answer whether the agent chose the wrong tool, entered a planning loop, or drifted away from its historical tool mix.
OpenTelemetry GenAI documentation in 2026 walks through exactly this span tree idea: an invoke_agent parent with chat and execute_tool children, plus token and duration metrics. Content capture is opt-in because prompts can contain sensitive data. Treat that as a design constraint, not an afterthought.
Keep your existing APM for the Nest/Express/Next services, queues, and databases around the agent. Add agent watch for the decision layer. Teams that try to force agent questions into only RED metrics (rate, errors, duration) end up with dashboards that look healthy while users complain that the bot "got stuck".
AgentPulse is deliberately narrow: loop detection, drift alerts, anomaly flags, and a replayable audit trail from pasted run observations or instrumented events. It is decision-support. It is not a formal security audit. It does not claim 100% detection. It does not claim you will never miss an incident.
Agent traces grow quickly if you store full prompts and tool payloads. Decide sampling and retention up front. Prefer hashes for arguments when you only need loop detection. Prefer short retention with export for incidents. Align access controls with GDPR Art. 32 thinking — security of processing — without pretending that a one-pager equals certification.
Emerging OTel discussions around agent authorization attributes stress an important honesty point: producer-emitted trust or drift scores are decision inputs for observation, not authenticated security facts you should blindly enforce on. Monitoring and authorization are different operations. AgentPulse watches behaviour; it does not replace a policy decision point that allows or denies a capability.
AgentPulse helps teams answer "what did the agent actually do" from a focused report rather than from five service log streams. Use it when you need a fast watch layer. Pair it with OTel if you already ship spans. Pair it with runtime guardrails if you need to stop loops, not just detect them. Pricing is $49/month or $490/year via dynamic checkout; Free remains labeled demo-capable.
A support agent normally calls a knowledge tool then a ticket tool. After a prompt tweak, it starts calling the knowledge tool five times with near-identical queries before ticking. Classic APM sees five fast successful calls. An agent watch report sees a rising repeat count and flags a loop candidate. The trail shows the identical query block. Engineering rolls back the prompt. That is the workflow we optimize for — evidence first, marketing second.
| Question | Prefer |
|---|---|
| Is the gateway slow? | Classic APM / RED metrics |
| Did the model call fail? | OTel GenAI spans + provider metrics |
| Did the agent loop on a tool? | Agent watch / AgentPulse |
| Should this capability be allowed? | Policy decision point / guardrail |
Mixing these layers in one dashboard is fine for executives; mixing them in one alert route is not. Page the team that can act. Gateway latency pages go to platform. Loop pages go to the agent owners. Authorization denials go to the security or policy owners.
You do not need a perfect schema on day one. A practical minimum is: runId, agentId, stepIndex, eventType (decision|tool|retry|final), toolName, argsFingerprint, resultClass, timestamp, modelId, promptHash. That set supports loop counts, drift of tool mix, and replay without storing every token.
When privacy policy allows content capture, store it behind stricter access and shorter retention than metadata. Default to metadata-only for routine runs; escalate to content capture only for sampled investigations.
Vendors that promise never-miss detection or 100% coverage are selling certainty that telemetry cannot provide. Ask for failure modes instead: what happens when events are dropped, when baselines are cold, when a new tool appears, when AI assist is offline. AgentPulse fails closed on live AI paths and labels demos. Prefer that posture over a silent mock that looks like success.
Week one: keep APM untouched. Add a runId to every agent invocation and log a single decision summary line per step. That alone lets you reconstruct rough trails from logs.
Week two: emit structured events for tool name and argument fingerprints. Build a weekly report of top repeat patterns per agent. No paging yet — just visibility.
Week three: define per-agent baselines for plan length and tool mix. Page only on extreme outliers with a linked trail. Tune thresholds with the on-call team before expanding coverage.
Week four: connect optional OTel GenAI spans if you already operate an OTLP pipeline. Keep AgentPulse (or equivalent) as the watch narrative even if raw spans live elsewhere. Dual sources are fine when roles are clear.
Throughout, refuse silent mock AI in any assistive report path. If the model key is missing, say so. If the upstream fails, say so. Operators trust systems that admit uncertainty more than systems that invent success.
Finally, write the honesty sentence into the product FAQ and security page: decision-support only, not a formal security audit, no 100 percent detection, no never-miss promise. That sentence protects customers and your future self during sales reviews.
If you only remember one sentence: classic APM watches services; agent observability watches decisions. You need both, assigned to the people who can actually change each layer.
Ship the trail first. Fancy dashboards without ordered evidence are theater. Ordered evidence without a dashboard still lets a careful engineer finish the incident.
You should — for the services around the agent. It will not show you tool choice, planning, or loops on its own.
OpenTelemetry gives you the plumbing for spans and context. The agent-specific part is deciding which events and attributes matter for loops and drift.
Behaviour changing over time — different tool mix, longer plans, more retries — without a code or prompt change that explains it.
Enough that sampling and retention need deciding up front, especially if you store prompts and tool outputs.
It can. Traces often contain user content; treat them with the same retention and access controls as your application data.
No. It is detection and evidence. Blocking harmful actions needs runtime guardrails.
No. Honesty rule: no 100% detection claims and no never-miss promises.
Try the product: AgentPulse
Application Performance Monitoring earned its keep by answering questions about services: is the checkout API slow, is the database saturated, did error rates spike after a deploy. Those questions still matter for the API gateway in front of your agent. They do not answer whether the agent chose the wrong tool, entered a planning loop, or drifted away from its historical tool mix.
OpenTelemetry GenAI documentation in 2026 walks through exactly this span tree idea: an invoke_agent parent with chat and execute_tool children, plus token and duration metrics. Content capture is opt-in because prompts can contain sensitive data. Treat that as a design constraint, not an afterthought.
Keep your existing APM for the Nest/Express/Next services, queues, and databases around the agent. Add agent watch for the decision layer. Teams that try to force agent questions into only RED metrics (rate, errors, duration) end up with dashboards that look healthy while users complain that the bot "got stuck".
Agent traces grow quickly if you store full prompts and tool payloads. Decide sampling and retention up front. Prefer hashes for arguments when you only need loop detection. Prefer short retention with export for incidents. Align access controls with GDPR Art. 32 thinking — security of processing — without pretending that a one-pager equals certification.
Emerging OTel discussions around agent authorization attributes stress an important honesty point: producer-emitted trust or drift scores are decision inputs for observation, not authenticated security facts you should blindly enforce on. Monitoring and authorization are different operations. AgentPulse watches behaviour; it does not replace a policy decision point that allows or denies a capability.