Architecture behind production AI agents

Deep Agents provides built-in capabilities for complex, multi-step tasks, including planning, context management, file access and delegation to subagents. It builds on LangGraph and can reduce the amount of supporting infrastructure a team needs to assemble.

In the Q&A, Srimanth recommends considering LangGraph when a workflow needs explicit control over its execution path. Deep Agents is useful for more autonomous tasks, but its planning and other supporting steps can consume additional tokens. Existing LangGraph projects do not automatically need to migrate.

Document retrieval gives an agent access to an indexed collection, while web grounding brings in information that may have changed since those documents were collected. In the example, Pinecone supports retrieval from internal documents and Tavily provides access to current web information.

The compliance audit agent uses these sources to compare regulatory updates with internal policies. This helps it work with information beyond the model’s training data, while evaluation remains necessary to check whether it retrieved and used the right evidence.

Traces show the sequence of model calls and tool executions within an agent run, including their inputs and outputs. They help developers inspect which information the agent retrieved, how it used tools and where execution time was spent.

In the demo, Srimanth inspects a compliance audit run and uses the waterfall view to investigate latency. Tracing explains what happened during a run; evaluators provide a separate check of whether the result meets the team’s quality requirements.

Start with a stable test dataset covering representative tasks and edge cases, with reference outputs that describe the expected results. Run the agent against those cases and apply consistent evaluators before comparing versions.

The session demonstrates a code evaluator that checks whether the agent’s predicted compliance level matches the reference label, alongside LLM-based evaluators for qualities such as relevance and completeness. LangSmith experiments let the team compare these scores together with latency and token usage when changing prompts, tools or models.

Measure cost per completed task alongside quality and latency. A cheaper model or a shorter response is only useful if the agent still completes the required work. Tool calls, repeated searches and planning steps also contribute to the overall cost.

In the case study, adding web grounding increased costs and the number of issues reported. The team then compared open models, including DeepSeek and NVIDIA Nemotron 3 Ultra, and reported lower costs after changing the model configuration. These are results for the demonstrated audit workflow, not a guaranteed saving for other agents. Ticket counts alone do not establish correctness; the evaluation results matter too.