Key Takeaways
- "Managed agent" names a responsibility boundary, not one standard product. A provider may run the model endpoint, the execution runtime, or the agent loop itself.
- Use a managed agent for multi-user production workloads when speed, isolation, scaling, and standard operations matter more than custom runtime control.
- Keep an agent local for interactive, single-user work near local files and tools. Build the cloud platform yourself only when its behavior is a product advantage or a hard requirement.
A managed agent platform can run code, scale workers, isolate sessions, store memory, connect tools, and trace execution. It cannot take responsibility for what the agent does with those capabilities.
The real decision is not whether an agent is "managed." It is where the provider's responsibility ends, where your product begins, and whether the workload needs a cloud service at all.
What is a managed agent?
In this article, "managed agent" describes which parts of an agent system a provider operates for you. This is a comparison model, not a universal vendor category.
A managed model endpoint serves inference requests. Your team runs the agent loop, tools, state, and runtime. Some products call richer agent services APIs. Classify them by what the provider operates, not by the word "API."
A managed runtime runs and scales your agent code. It may also isolate sessions and provide identity and traces, while your team controls the loop.
A managed harness runs the runtime and agent loop from the model, instructions, tools, memory rules, and limits you supply. Your team still owns permissions and outcomes.
| Managed layer | Provider typically operates | Your team still owns |
|---|---|---|
| Model endpoint | Model serving | Agent loop, runtime, state, tools, and operations |
| Agent runtime | Execution and scaling; often sessions, identity, traces | Agent loop, tool behavior, and product logic |
| Agent harness | Runtime plus a configured agent loop | Permissions, evaluation, escalation, and user outcomes |
AWS calls the highest layer in this model a "managed agent harness," but the term is not an industry standard.
Providers expose different slices of this stack. This is a responsibility model, not a claim of feature parity.
AWS exposes the boundary directly: AgentCore Runtime leaves orchestration to customer code, while AgentCore harness supplies the loop. Microsoft separates configuration-driven prompt agents from Hosted agents, which run customer code behind managed endpoints. Google's Agent Platform combines a managed runtime with session, memory, evaluation, observability, and sandbox services.
The products are not equivalent. They show that "managed" can stop at different layers.
A production run crosses more than the model endpoint
A chat demo can look like a single model request. A production agent run is a longer chain.
The application authenticates the caller and creates or resumes a session. The runtime loads the agent version and state. The loop calls the model, tools, or memory, but every tool call still needs authorization appropriate to the action.
Runs can pause, retry, or resume on another worker. Traces connect the steps, while recovery accounts for actions that already happened.
A conceptual production lifecycle. Approval is conditional. Authentication, authorization, evaluation, and escalation remain product decisions even when the provider runs the runtime.
A managed runtime can supply execution, and a harness can also supply the loop. Neither knows whether a refund, merge, or email is correct. Policy and recovery remain application responsibilities.
The upside: less infrastructure before production
- Less platform work: You may avoid building schedulers, session isolation, identity plumbing, tracing, and deployment controls.
- Shared operating conventions: Common paths for versions, sessions, and traces can make incidents easier to investigate.
- Elastic execution: A managed runtime can add and retire capacity without your team operating a scheduler.
- Earlier behavior feedback: Built-in logs and lifecycle controls let engineers start measuring agent behavior sooner.
These benefits appear only when platform defaults match the workload. Workarounds quickly return the operating burden.
The downside: coupling reaches beyond the SDK
- State is hard to move: Replacing an API call may be straightforward. Reproducing how a provider resumes sessions, stores memory, or versions a running agent may not be.
- Control narrows as management expands: Custom orchestration, hooks, protocols, networking, or resource shapes may not fit the supported path.
- Limits can become product limits: Quotas, regional availability, and preview features can constrain what your agent can offer.
- Cost has several meters: A completed task can consume tokens, tool calls, stored state, and active compute across concurrent sessions. Microsoft, for example, documents active CPU and memory billing for Hosted agent sessions.
- Infrastructure controls are not product authorization: Managed identity and session isolation help, but your team still decides who may invoke a tool, what it may read, and when a person must approve an action.
Measure cost per completed task, including retries and failed runs, rather than cost per request. Test how the platform behaves when a tool times out, state cannot be resumed, or the wrong action succeeds.
When to use managed, self-built, or local agents
The right operating model depends on where the work happens and who must rely on it.
Use a local agent when the workstation is the product boundary
Here, local describes where the agent loop and tool execution run, not necessarily inference. A local agent may still call a hosted model.
A local agent fits interactive work by one person: editing a repository, searching notes, transforming files, or operating workstation tools. File and tool operations can remain on the workstation, though prompts or retrieved content may still leave it when the agent calls a hosted model.
It also inherits the workstation's permissions and availability. Move beyond local execution when the agent must serve many users, run unattended, accept external events, or provide central audit controls.
Use a managed agent when the workload needs a service
A managed runtime or harness is a practical default for a multi-user product when the loop is not the main technical advantage.
It fits bursty workloads, long sessions, background jobs, and tools that need a stable cloud identity. The team avoids building scheduling, sandboxing, session, or tracing capabilities only when the selected service supplies them.
The fit weakens when a critical requirement falls outside the supported path, such as a custom sandbox, unusual hardware, private networking, or hard portability. One workaround may be manageable. A product made mostly of workarounds has the wrong boundary.
Build the platform when runtime behavior is part of the advantage
Self-building means owning packaging, scheduling, isolation, identity, observability, retries, upgrades, and rollback.
That investment can make sense when orchestration is core intellectual property, required controls are unavailable, the workload needs unusual hardware or networking, or measurements show that custom resource management justifies the operating cost. It also requires a platform team that will not starve the product.
Do not self-build to avoid the appearance of lock-in. A portability layer over models and tool schemas may be cheaper. Build because a measured requirement demands control.
One agent across three operating models
Consider an illustrative code-review agent. Keep it local while one engineer runs it against a checkout. Move it to a managed runtime when repository webhooks must be processed for every team, jobs must survive laptops sleeping, and traces must be centralized. Build the runtime only if repository execution requires a sandbox or network boundary the managed service cannot provide.
The agent did not become more advanced. Its operating requirements changed.
The operating models overlap. Start with the simplest one that meets the workload's availability, control, and governance requirements.
| Operating model | Best fit | Main strength | Main limit |
|---|---|---|---|
| Local agent | One user, local files and tools, interactive work | Proximity and fast iteration | Weak fit for shared, always-on production |
| Managed agent | Multi-user services, bursty or long-running sessions | Production infrastructure without a fleet | Provider constraints and deeper state coupling |
| Self-built | Differentiated runtime or hard control requirements | Maximum control over execution | Platform staffing and continuous operating cost |
References
- AWS, AgentCore harness vs. Runtime.
- Google Cloud, Scale your agents.
- Microsoft, What is Microsoft Foundry Agent Service?.
- Microsoft, Hosted agents in Foundry Agent Service.