Guide

AI agent architecture: the parts and how they fit together

Harrowtech.ai · last checked 13 October 2026

AI agent architecture comes down to seven parts. A model does the reasoning. Instructions set its job. Tools let it look things up and take actions. Memory and context hold what it knows so far. A loop repeats decide, act and check until the job is done or a limit is reached. Guardrails, including human approval, control which actions it may take. Logs and tests show what it did and how well. They fit together around the loop: on each turn the model reads its instructions, context and tool list, asks for a tool, the guardrails allow or stop the request, the tool runs, and its result goes back to the model for the next decision.

The grouping into seven is this guide’s own. Anthropic, Google Cloud and Microsoft describe the same parts in their documentation under different names, and the tables below give their words. For what makes a system an agent rather than a workflow, see AI agents vs workflows vs agentic AI; this page assumes that and looks inside.

The seven parts

ModelReads the situation and chooses the next step. Google Cloud’s architecture guide calls it “the core reasoning engine that powers your agent’s decision-making capabilities.” Anthropic’s Claude Code docs say the loop is “powered by two components: models that reason and tools that act.”
InstructionsThe standing brief, usually called the system prompt. In Google Cloud’s words, it “shapes the agent’s behavior by defining its core task, persona, and operations, and the specific conditions for using each tool.”
ToolsThe connections that let the agent read data and change things in other systems. Anthropic: “Without tools, Claude can only respond with text.” Each tool carries a written description, and Google Cloud notes that the model “uses this description to reason about when the tool is needed, what inputs to provide, and how to interpret the output”.
Memory and contextWhat the agent can see while it works, and what it keeps between runs. Google Cloud separates short-term memory, the state of one session including “the results of any tool calls”, from long-term memory, “a persistent knowledge base that exists across all conversations”. Anthropic notes that the context window “does not reset between turns within a session”.
The loopThe cycle of deciding, acting and reading the result. Anthropic describes agents as “typically just LLMs using tools based on environmental feedback in a loop.” The loop needs a way to end: Anthropic mentions “stopping conditions (such as a maximum number of iterations)”.
Guardrails and approvalThe rules for which actions run on their own, which wait for a person, and which never run. Anthropic’s Agent SDK lets you “auto-approve specific tools, block others entirely, or require approval for everything.” Google Cloud’s human-in-the-loop pattern pauses the agent “at a predefined checkpoint” until a person has reviewed its work.
Logs and evaluationThe record of what happened, and the tests that show whether the agent does the job well. Microsoft’s guidance: “Instrument all agent operations and handoffs.” Anthropic’s SDK ends each run with a result that includes “token usage, cost, and session ID”.

Google Cloud’s plain-English page on agents lists a shorter set of four: persona, memory, tools and model. Its architecture guide adds the pieces needed to run an agent in production, covered further down under where the parts run.

How the parts fit together: one turn of the loop

Anthropic’s Agent SDK documentation sets out the cycle step by step. Read with the parts above, one turn looks like this:

   task + instructions + context so far
                   |
                   v
          +------------------+
          |  model decides   |<-----------------+
          +------------------+                  |
             |            |                     |
       final answer   tool request              |
             |            |                     |
             |            v                     |
             |   +------------------+  blocked: |
             |   | guardrail check  |--refusal->+
             |   | allow/ask/block  |           |
             |   +------------------+           |
             |            | allowed             |
             |            v                     |
             |   +------------------+           |
             |   |    tool runs     |--result-->+
             |   +------------------+
             v
   run recorded: answer, cost, usage
1. The task arrivesThe model receives the request “along with the system prompt, tool definitions, and conversation history.” (Anthropic)
2. The model decidesIt “may respond with text, request one or more tool calls, or both.”
3. The guardrails checkBefore a tool runs, permission rules decide whether it runs, waits for a person, or is refused. Hooks can “intercept, modify, or block tool calls before they run.” A refused call is not the end: the model “receives a rejection message as the tool result and typically attempts a different approach or reports that it couldn’t proceed.”
4. The tool runsIts result is added to the context. Anthropic’s engineering post stresses that agents need “‘ground truth’ from the environment at each step (such as tool call results or code execution) to assess its progress.”
5. Repeat or stopThe loop goes round again until the model replies with no tool calls. It can also be stopped by a cap on turns or on spend. In the Agent SDK both caps default to “No limit”, and the docs say “Setting a budget is a good default for production agents.”
6. The result is recordedThe SDK reports whether the run succeeded or hit a limit, with cost and usage. Hooks can also fire after each tool for auditing.

Two consequences matter when you commission an agent. First, context grows with every turn. Anthropic says “Everything accumulates”, tool outputs included, and when the context nears its limit the SDK summarises older history, warning that “specific instructions from early in the conversation may not be preserved.” Standing rules therefore belong in the instructions, which are sent with every request, not in the opening message. Second, the guardrail check sits outside the model. Anthropic’s hooks “run in your application process, not inside the agent’s context window”, so a rule written there does not depend on the model choosing to follow it.

Where the parts run

Google Cloud’s architecture guide lists eight components for a production agent: a frontend (the screen people use), an agent development framework, tools, memory, a design pattern, an agent runtime, the AI model and a model runtime. Two of them are about hosting. The agent runtime is where your own logic runs, the model runtime is where the model is served, and “From this runtime, your application makes API calls to the models that your chosen model runtime hosts and manages.”

The split matters for data. The agent’s logic, its memory and its logs can sit in one place while the model is reached over an API from a provider. Google Cloud adds that “Workloads with strict security and data residency requirements also require a self-hosted model, because it lets you run the model within your own network.” Our guide to whether AI automation is safe covers what the main providers say they keep.

Start with the least agent that works

All three companies say to begin small. Microsoft sets out three levels and says to “Use the lowest level of complexity that reliably meets your requirements.”

Direct model call“A single language model call with a well-crafted prompt. No agent logic, no tool access.” Enough for classifying, summarising or translating in one pass.
Single agent with toolsOne agent that reasons and picks from its tools, looping through model calls and tool use. Microsoft calls it “Often the right default for enterprise use cases” and adds: “To guard against infinite tool-call loops, set iteration limits.” Google Cloud also recommends starting here for teams early in agent development.
Several agentsSpecialised agents that work together, managed by an orchestrator or a peer-based protocol. Microsoft says this “Adds coordination overhead, latency, and failure modes”, justified when a single agent cannot reliably handle the task because of prompt complexity, too many tools or security requirements.

Below the first level sits plain code. Microsoft’s Agent Framework documentation puts it bluntly: “If you can write a function to handle the task, do that instead of using an AI agent.” Anthropic’s engineering post makes a similar point: for many applications, “optimizing single LLM calls with retrieval and in-context examples is usually enough.”

The common patterns, under three names

Each company has its own catalogue of patterns. They overlap heavily, and Microsoft’s page lists several of the other names itself. The table lines them up. Where it says “closest”, the descriptions overlap without being the same; that matching is ours.

The ideaAnthropicGoogle CloudMicrosoft
Fixed steps in order, each using the last one’s output. Path set in code.Prompt chainingSequentialSequential orchestration (“also known as a pipeline, prompt chaining, or linear delegation”)
Several pieces of work at once, results combined.ParallelisationParallelConcurrent orchestration
One drafts, another checks against criteria, round again until it passes or hits a cap.Evaluator-optimiserReview and critique; iterative refinementMaker-checker loop (“also known as evaluator-optimizer”)
Sort the request, then pass it to the right specialist.RoutingCoordinator (closest)Handoff (“also known as routing, triage, transfer, dispatch, or delegation”)
A lead model splits the job, hands out parts and pulls the results together.Orchestrator-workersCoordinator; hierarchical task decompositionMagentic orchestration (closest)
One model reasons, uses a tool, reads the result, repeats.The agent loopSingle-agent system; ReAct (reason and act)Single agent with tools
Stop at a set point for a person to approve or correct.Pausing “for human feedback at checkpoints”Human-in-the-loopHuman participation (HITL)

The useful split is who chooses the path. Google Cloud groups its patterns into “Workflows that are deterministic”, where the sequential and parallel patterns run “without having to consult an AI model”, and “Workflows that require dynamic orchestration”, where a model decides. It says the sequential pattern “can reduce latency and operational costs” compared with model-led orchestration, at the price of flexibility. Microsoft lists the mistake in both directions: “Using deterministic patterns for workflows that are inherently nondeterministic”, and the reverse. Many tasks need both. Google Cloud’s custom logic pattern covers that case, for workflows that need to “mix predefined rules with model reasoning”: code for the branches you can predict, a model for the parts you cannot.

For Anthropic’s own explanation of its five workflow patterns, and its products for building agents, see Anthropic AI agents.

Guardrails and human approval in practice

Guardrails come in layers, and the NCSC’s advice favours the ones written as ordinary code over requests made in the prompt. The NCSC, writing about prompt injection, says protections “need to therefore focus more on deterministic (non-LLM) safeguards that constrain the actions of the system, rather than just attempting to prevent malicious content reaching the LLM.” Its reason: when a model can call tools, the worst case becomes “whatever the worse case scenario would be of giving an attacker direct access to those tools/APIs”.

Microsoft adds a practical detail: a mandatory approval makes the run wait at that step, so the agent should save its state there and carry on afterwards without redoing earlier work.

Logs and testing

An agent can take a different route each time it runs, which changes how you check it. Microsoft: “Agent outputs are nondeterministic, so use scoring rubrics or language-model-as-judge evaluations rather than exact-match assertions.” It also recommends tracking performance and token use per agent so you can see where cost goes.

The NCSC treats logs as a security tool as well. It suggests “logging enough information to identify suspicious activity, potentially including full input and output of the LLM, as well as tool use, API calls etc.”, and notes that watching for failed tool or API calls “could identify earlier stages of an attack.”

Anthropic’s advice is to measure and adjust: “The key to success, as with any LLM features, is measuring performance and iterating on implementations.” One practical way to do that for a small business: keep a set of real past cases with known right answers, and run them before go-live and again after any change to the model, the instructions or the tools.

Worked example: a lead finder, part by part

Ovais Khan, Commercial Director of saifcarpets.com, asked Harrowtech.ai for a lead generation agent. This is his review from 4 June 2026, as it appears on our home page:

“Reached out to harrowtech.ai for a custom lead generation agent. It runs every day, finds leads, qualifies them, adds as much info as possible and creates a spreadsheet with all of the info in it. Ready to go with a pre-written outreach email and I just review and set status to send; that’s it. Very happy with the new business I have picked up.”Ovais Khan, Commercial Director, saifcarpets.com

Set against the seven parts, the review shows some and is silent on others:

How it starts“It runs every day”: it works on a daily cycle.
Tools and actionsIt “finds leads”, “adds as much info as possible” and “creates a spreadsheet”. So it reaches outside sources and writes to a spreadsheet. The review does not name the sources.
Judgement stepsIt “qualifies” leads, and an outreach email is “pre-written”. The review does not say which steps use AI and which use fixed rules.
Where results are keptThe spreadsheet, “with all of the info in it”.
Human approval“I just review and set status to send”. Sending waits for him.
Model, build tool, logsNot described in the review, so not described here.

The approval point is the design choice that shapes the rest. Because sending waits for his status change, the agent does the finding and the preparing while the decision to send stays with the owner.

What to decide before you build one

Google Cloud starts its design process with four questions: can the task be done in predefined steps or is it open-ended, how fast must it respond, what can you spend on model calls, and does it involve “high-stakes decisions, safety-critical operations, or subjective approvals that require human judgment?” Turned into a brief for whoever builds it, each part needs a decision:

  1. The job and the finish line. One sentence on what the agent does, and how you will know a run succeeded. This becomes the instructions and the stopping rule.
  2. Which steps are fixed. Mark every step that can follow a rule. Those become code, not model decisions.
  3. What it may read and what it may change. List the systems and the actions in each. This is the tool list, and it should be the shortest one that does the job.
  4. Who approves what, and where. Name the actions that wait for a person and the place the person approves them.
  5. What it remembers. Decide what is kept between runs and where. Microsoft advises scoping saved state “to the minimum necessary information” to reduce token overhead and privacy risk.
  6. Limits and failure. Set a cap on steps and spend, and say what happens when a cap is hit. Microsoft’s examples of fallback behaviour include “escalation to a human reviewer or a quality warning alongside the best possible result.”
  7. Logs and tests. Decide what is recorded, who looks at it, and which past cases it must pass before it goes live.
  8. Where it runs and who owns it. Whose account holds the agent, its keys and its data, and what happens if you change supplier.

Our guide on how to automate your business with AI covers picking the job in the first place, and AI automation cost covers the bill.

How we build them

Our builds put fixed rules first: any step that can run on rules does, with no AI. AI goes in only where it helps, and nothing leaves the business until a person has approved it. We build in n8n for speed and flexibility, so projects get moving fast. What we build lives in your own account, in your name, with documentation, and keeps running if we part ways. A first automation starts from £950. See our services or get in touch.

What this guide does not cover

It does not teach you to code an agent or compare frameworks such as Anthropic’s Agent SDK, Google’s ADK and Microsoft’s Agent Framework feature by feature. It does not go into the multi-agent patterns beyond their names, or recommend a model. Prices are in AI automation cost, data protection law in AI automation and GDPR, and the build tools in n8n, Zapier and Make compared.

Further reading

Sources

All read 13 October 2026.

Related: All guides · Harrowtech.ai home