• Explainer
  • E-commerce
  • 5 min read

AI Performance, Explained: What the Agents Did, How Reliably, and What It Cost

A budget guard on the AI itself, metered from its own traces.

Author
ftes.ai Editorial
ftes.ai Team

The short answer

The AI performance surface in the commerce engine reports what the platform's own agents did, how fast and how reliably they did it, and what they cost, metered from the agents' own execution traces with a budget guard. It is the engine holding itself to the same measurement standard it applies to the business.

Put a Digital FTE on thisEvery role proposes with evidence and reports to you.

Key takeaways

  • Latency, reliability and cost are measured per agent and per surface from real traces.
  • A budget guard bounds token spend and reports against it rather than after it.
  • Failures are shown as failures, including the ones that produced no output.
  • Model vendors are configurable cards, so cost and reliability can be compared across providers.
  • On ftes.ai the same discipline appears as the token wallet: balance visible, hard stop at zero.

What does this surface report?

Per agent and per surface: how many runs, how long they took, how many succeeded, how many failed and why, and what they consumed. The Morning Brief's writer, the chat analyst, the article generator and the social drafter each appear separately, because their costs and their failure modes are different.

The numbers come from the agents' own traces, so they are what actually happened rather than an estimate applied after the fact.

  • Runs per agent and per surface, with durations and success rates.
  • Failures with their reasons, including runs that completed but produced nothing.
  • Token consumption per agent, metered from the agents' own traces.
  • Consumption against the budget you set, as it happens rather than afterwards.
  • Latency and reliability per model provider, so a vendor choice can be compared on evidence.

Why meter the AI at all?

Because AI is a variable cost that can quietly grow. An agent that retries silently, a surface that regenerates more often than it needs to, or a model change that doubles token use are all invisible without metering, and they show up as a bill rather than as a decision.

Metering also makes model choice an evidence-based decision. When two providers are configured, their latency, reliability and cost on the same work can be compared rather than argued about.

It also settles arguments about whether a surface earns its cost. An agent that writes a weekly report for a few cents is obviously worth keeping; one that regenerates an expensive draft every time a page opens is a design problem the metering finds long before the invoice does.

What does the budget guard do?

The brand sets a budget. The guard reports consumption against it as it happens, warns as it approaches, and behaves predictably at the limit rather than failing at an arbitrary moment. Nothing about the engine's spending on AI is a surprise at the end of the month.

This is the same principle as the money-safety rule in advertising, applied to the platform's own consumption: bound it, show it, and make the limit real.

Failures are shown as failures

An agent run that produced nothing is recorded as a failed run with its reason, not hidden because the surface recovered gracefully. A brief that could not be written because a source was degraded appears here as well as on the brief itself.

That is honest data applied to the platform's own behaviour, and it is what lets an owner tell the difference between a quiet morning and a broken one.

Why are model vendors swappable?

Text, image and video providers are configurable cards rather than hard-coded dependencies: one default for text, one for images, one for video. The engine is vendor-independent by design, and this surface is where the consequences of a vendor choice become visible in latency, reliability and cost.

For a brand, the practical benefit is that a provider outage or a price change is a configuration decision rather than a platform crisis.

How does it relate to the token wallet on ftes.ai?

On ftes.ai, a plan is a block of metered work for the whole organization: the balance is visible next to the work, and at zero the roles pause and say so rather than overdrawing. The engine's AI performance surface is the same idea seen from inside the platform: consumption measured at the point it happens, against a bound the brand set.

Between them, an owner can always answer two questions: what did the AI do this week, and what did it cost to do it.

When should an owner look at this surface?

When a surface feels slow, when a brief did not arrive, or when the monthly consumption moved without an obvious reason. Those three questions are all answered here, and each of them has a per-agent answer rather than a platform-wide one.

Otherwise it is a surface to check monthly. The budget guard reports as it goes, so the routine reading is the trend: which agents are getting more expensive, which are failing more often, and whether a provider change would be worth testing.

Frequently asked questions

Where do the AI metrics come from?

The agents' own execution traces, so they reflect what actually ran rather than an estimate.

What does the budget guard do?

Reports consumption against a budget you set, warns as it approaches, and behaves predictably at the limit.

Are failed runs shown?

Yes, with their reasons, including runs that produced no output.

Can I change model providers?

Yes. Text, image and video vendors are configurable cards; the platform is vendor-independent by design.

How does this relate to ftes.ai plans?

Plans are token-metered blocks of work with a visible balance and a hard stop at zero; this surface is the same discipline inside the engine.

  • E-commerce
  • Commerce engine
  • Digital FTE
Further reading

Keep going.

Related guides on hiring and managing Digital FTEs.

Ready to hire your first Digital FTE?

Pick a role, connect the tools it needs, and review its first proposals this week.