CH14-01: AI and Agentic Architecture Mental Model

Goal

Introduce AI and agentic architecture as controlled orchestration around models, data, tools, identity, safety, evaluation, and operations.

AI systems can look deceptively simple from the outside. A user asks a question, the application sends a prompt to a model, and a response comes back. That is enough for a demo, but it is not enough for an architecture.

The architectural question is broader:

What is the model allowed to see, decide, call, return, and prove?

Once that question is clear, AI architecture starts to look less mysterious. It becomes a familiar cloud design problem with a new kind of capability in the middle.

The Short Version

AI architecture is the design of a controlled system around model capability, not simply a model call.

A model can generate text, reason over instructions, summarize context, classify input, create structured output, or help decide which tool to call. The application around the model still owns identity, authorization, data access, workflow, validation, logging, cost, and incident response.

Agentic architecture adds another layer of responsibility. In a simple Q&A flow, the system usually answers from a prompt and some context. In an agentic flow, the system may plan steps, choose tools, call APIs, inspect results, and continue until a task is complete or a boundary stops it. That can be powerful, but it also means the architecture must define the boundaries before the workflow acts.

The Responsibility Model

Think of an AI system as a workflow with a model inside it, not as a model with a small application attached.

flowchart LR
    User["User<br/><small>Who is asking?</small>"]
    App["Application or Agent Layer<br/><small>What is the task?</small>"]
    Context["Retrieval and Context<br/><small>What data is allowed?</small>"]
    Model["Model Endpoint<br/><small>What capability is used?</small>"]
    Tools["Tools and APIs<br/><small>What actions are allowed?</small>"]
    Controls["Safety, Evaluation, Monitoring<br/><small>What can we prove?</small>"]

    User --> App
    App --> Context
    Context --> Model
    Model --> App
    App --> Tools
    App --> Controls
    Tools --> Controls
    Model --> Controls

This diagram is deliberately simple. Different products and frameworks may arrange the pieces differently, but the responsibilities keep appearing:

Do not worry if some terms are new. Retrieval, grounding, tools, evaluation, and agent orchestration will appear throughout the chapter. For now, focus on the job each part performs.

What Changes When an Application Uses a Model

A traditional application usually follows code paths the team wrote explicitly. It still has bugs and edge cases, but the core behavior is deterministic enough that testing can focus on known inputs, outputs, and failure paths.

An AI application introduces model behavior that is flexible, probabilistic, and shaped by prompts, context, model version, tool definitions, and user input. That flexibility is the value, but it also changes the architecture conversation.

The team now has to design for questions such as:

• Which data can be sent to the model?

• Which data should be retrieved and included as grounding context?

• Which actions can the model or agent request?

• Which actions require approval before execution?

• How will the team evaluate answer quality and safety?

• How will incidents be investigated when the system gives a wrong answer or calls the wrong tool?

• How will cost be controlled when usage grows?

These are not only AI questions. They are architecture questions about boundaries, ownership, trust, and operations.

What Makes a Workflow Agentic

A workflow becomes agentic when the system can do more than produce a single answer. It may choose steps, call tools, inspect outputs, and continue toward a goal.

For example, a simple assistant might answer:

The travel policy says economy class is required for domestic trips.

An agentic workflow might do more:

1. Interpret the employee's request.

2. Retrieve the travel policy.

3. Check the employee's role and trip details.

4. Call an expense-system API to start a request.

5. Ask for approval if an exception is needed.

6. Log the action and return a confirmation.

That second workflow crosses into enterprise action. The model is no longer only helping with language. It is participating in a business process. The architecture must decide which parts are automated, which parts require human review, which identity is used, and how each step is audited.

Example: Internal Support Assistant

Imagine an internal assistant that answers employee questions and creates support tickets.

The early demo might look like this:

Employee question -> model -> answer

The production design needs more structure:

Employee identity
  -> assistant application
  -> authorization-aware retrieval from policy documents
  -> model response with source references
  -> optional ticket creation through an approved API
  -> logs, feedback, evaluation, and cost tracking

The difference is not just technical polish. The production design can answer the questions a real organization will ask:

• Did the assistant use approved policy sources?

• Did the employee have permission to see the retrieved content?

• Did the ticket API validate the request before creating a record?

• Can support teams audit what happened?

• Can the product team evaluate whether answers are accurate?

• Can finance or platform teams detect cost spikes?

That is the point of the mental model. A useful AI architecture is not measured only by how fluent the answer sounds. It is measured by how well the system controls and proves the workflow around that answer.

Common Mistakes

• Treating AI as only a chatbot UI. The chat surface is only the entry point. The real architecture includes data, identity, orchestration, tools, evaluation, monitoring, and ownership.

• Letting the model see everything the application can access. Application access is not the same as user-specific authorization. Retrieval and tool calls still need permission checks.

• Giving tools broad credentials. A tool should expose a narrow approved action, validate inputs, use the right identity, and produce an audit trail.

• Skipping evaluation until after launch. Teams need representative test cases, safety checks, regression checks, and quality thresholds before users depend on the system.

• Ignoring cost behavior. Token use, retrieval, embeddings, logging, evaluation, and tool execution can grow quickly when adoption increases.

If Someone Asks You

AI architecture in Azure is the design of the controlled workflow around model capability. It defines who is asking, what data can be retrieved, which model endpoint is used, which tools can be called, which safety and evaluation controls apply, and how the system is monitored and operated.

What to Remember

Design what the model can see, decide, call, return, and prove.

Questions to Check Understanding

1. What responsibilities surround a model call in a production AI application?

2. What makes an agentic workflow riskier than a simple Q&A flow?

3. Why are evaluation and monitoring architecture concerns rather than optional review steps?