# AI agents explained: tools, loops and when not to use one

Updated: 2026-11-13. https://thegccengineer.com/guides/ai-agents-explained

**Quick answer:** An AI agent is an LLM application in which the model decides, step by step, which tool to call next and when the task is finished. The surrounding program runs each tool, returns the result to the model and repeats until the model gives a final answer or a limit is reached. Agents suit open-ended tasks; a fixed workflow is cheaper and more reliable when the steps are known in advance.

An AI agent is an LLM application where the model, not the programmer, decides the next step. The model is given a goal and a set of tools, chooses which tool to call, reads the result and repeats until it judges the task complete. That is the whole idea behind AI agents explained without the marketing: a model inside a loop, with tools. The loop gives flexibility on open-ended tasks and takes away predictability, which is why knowing when not to use an agent matters as much as knowing how to build one.

Agents come late in the [AI engineer roadmap](https://thegccengineer.com/guides/ai-engineer-roadmap) because they depend on LLM APIs, prompting and retrieval. If retrieval is new to you, read [how RAG works](https://thegccengineer.com/guides/rag-explained) first.

## What is an AI agent?

An AI agent is a system made of three parts: a language model that reasons about what to do, a set of tools the model may ask to use, and a control loop that runs those tools and feeds the results back. The defining feature is that the sequence of steps is chosen at run time by the model.

Compare two ways of handling a refund request. In a fixed program, the code always looks up the order, checks the policy and then drafts a reply. In an agent, the model is told it has an order lookup tool, a policy search tool and a refund tool, and it works out which to call and in what order for each case, including the unusual ones nobody planned for.

## How does tool use work?

Tool use works by the model producing a structured request that your program then executes; the model itself never runs anything. You describe each tool to the model with a name, a plain-language description and a schema for its arguments. When the model decides a tool would help, it returns the tool's name and arguments instead of a text answer. Your code validates the request, runs the function and sends the result back as the next message.

Two consequences follow. First, tool descriptions are prompts: a vague description produces wrong calls, and two tools with overlapping purposes confuse the model. Second, your code sits between the model and every action, which is where all real control lives. The model can ask to delete a record; whether that happens is your program's decision.

## How does the agent loop work?

The agent loop works by alternating model calls and tool executions until the model returns a final answer or a limit stops it.

**One pass through the agent loop**

1. **Receive the task**: The goal, the conversation so far and the list of available tools go to the model.
2. **Decide**: The model either answers directly or requests a tool with specific arguments.
3. **Check**: Your code validates the request: known tool, valid arguments, permitted for this user.
4. **Act**: The tool runs and returns a result or an error message.
5. **Observe**: The result is added to the conversation so the model can see what happened.
6. **Repeat or stop**: The loop continues until a final answer, a step limit, a budget limit or a request for human help.

A minimal version of that loop is short. In the sketch below, `call_model` stands for whichever model API you use and returns a reply that either contains text or a tool request.

```python
def run_agent(task: str, tools: dict, call_model, max_steps: int = 8) -> str:
    messages = [{"role": "user", "content": task}]
    for _ in range(max_steps):
        reply = call_model(messages, tools)
        if reply.tool_call is None:
            return reply.text  # the model has finished
        name, args = reply.tool_call.name, reply.tool_call.args
        if name not in tools:
            result = f"error: unknown tool {name}"
        else:
            try:
                result = tools[name](**args)
            except Exception as exc:
                result = f"error: {exc}"
        messages.append({"role": "assistant", "tool_call": reply.tool_call})
        messages.append({"role": "tool", "name": name, "content": str(result)})
    return "stopped: step limit reached"
```

Notice what the sketch does with failures. An unknown tool or a raised exception becomes a message back to the model, which can then correct itself, and the step limit guarantees the loop ends. Planning, memory and reflection, the terms you will meet in agent literature, are refinements of this loop: asking the model to write a plan first, storing notes between steps, or having it review its own work before finishing.

## What is the difference between an agent and a workflow?

The difference is who decides the sequence of steps. In a workflow, the developer fixes the path in code and the model fills in individual steps, such as classifying a ticket or drafting a reply. In an agent, the model chooses the path. Many production systems described as agents are workflows with one or two model-driven decisions, and that is usually a sound design.

**Workflow or agent?**

**Fixed workflow**

- Steps are known in advance
- Predictable cost and latency
- Each step can be tested in isolation
- Fails in known places
- Cannot handle cases nobody planned for

**Agent**

- Steps depend on what is found along the way
- Cost and latency vary per run
- Must be evaluated on whole tasks
- Can fail in new ways each run
- Adapts to unusual cases

## When should you not use an agent?

Do not use an agent when you can write down the steps beforehand. If the task can be drawn as a flowchart, a workflow will be cheaper, faster, easier to test and easier to explain to a risk team. Reach for an agent only when the path really does depend on intermediate results that cannot be anticipated.

The underlying reason is that errors compound. Each step has some chance of going wrong, and a run succeeds only if enough steps go right, so a long chain is noticeably less reliable than any one of its links. Every additional step also adds a model call, which adds cost and delay.

| Situation | Better choice | Reason |
| --- | --- | --- |
| Extract fields from a document, then validate them | Workflow | The steps never change |
| Answer questions from a document collection | RAG, a fixed pipeline | Retrieve then generate is enough |
| Route a request to one of several handlers | Workflow with a model as router | One decision, then fixed paths |
| Investigate an incident across logs, metrics and code | Agent | The next step depends on each finding |
| Research a question across many unknown sources | Agent | The path cannot be planned ahead |
| Any action that moves money or deletes data | Workflow or agent with human approval | The cost of an error is too high to automate fully |

*Matching the design to the task.*

## What guardrails does an agent need?

An agent needs guardrails enforced in code, because instructions in a prompt are requests and not controls. The essential ones are:

- **Limits.** A maximum number of steps, a time limit and a spending limit per run, so a confused agent cannot loop indefinitely.
- **Least privilege.** Give each agent only the tools it needs, with the narrowest permissions. Prefer read-only tools; scope write tools to the current user's own data.
- **Argument validation.** Check every tool call against its schema and against business rules before running it.
- **Human approval.** Require confirmation for actions that are irreversible or costly: payments, deletions, messages sent to customers.
- **Untrusted input handling.** Text returned by tools, such as web pages, emails and retrieved documents, may contain instructions written to hijack the agent. This is prompt injection. Treat tool output as data, and never let it expand what the agent is allowed to do.
- **Tracing.** Record every model call, tool call and result. Without a trace, a failed run cannot be diagnosed.

> **The dangerous combination:** An agent that can read private data, read content from outside the organisation and send information out has everything an attacker needs. Remove one of the three, or put a human approval step between them.

## What do interviewers ask about agents?

Interviewers ask about agents to find out whether you have judgement, not whether you know a framework. Common prompts are to design an agent for a concrete task, to explain how you would stop it doing damage, and to say how you would measure whether it works. Expect a follow-up asking why the task needs an agent at all; a candidate who argues for a simpler workflow where one fits usually scores higher than one who reaches for autonomy by default.

Measuring an agent means scoring whole tasks on a fixed test set, covered in [how to evaluate an LLM application](https://thegccengineer.com/guides/llm-evaluation-basics). Typical questions with answer structures are in [GenAI interview questions](https://thegccengineer.com/guides/genai-interview-questions). Agents and tool use are core skills for the [GenAI and LLM engineer role](https://thegccengineer.com/guides/genai-llm-engineer-role) and the [AI backend engineer role](https://thegccengineer.com/guides/ai-backend-engineer-role); you can compare what each track expects in the [role explorer](https://thegccengineer.com/roles).

## Key takeaways

- An agent is a loop: the model chooses an action, your code runs it, and the result goes back to the model.
- The model never runs a tool itself; it only asks, and your program decides whether to carry the request out.
- Errors compound across steps, so long agent runs are less reliable than their individual steps suggest.
- Guardrails belong in code, not in the prompt: step limits, permissions, validation and human approval.
- If you can draw the steps as a flowchart, build a workflow instead of an agent.

## Frequently asked questions

### Is a chatbot an AI agent?

Not by itself. A chatbot that only replies with text is a single model call per turn. It becomes an agent when the model can choose to call tools, such as searching a knowledge base or looking up an order, see the results, and decide what to do next before answering.

### Do I need an agent framework to build an agent?

No. The core loop is a few dozen lines of code around a model API that supports tool calls. Writing it once by hand is the best way to understand what frameworks do. Frameworks become useful for state persistence, retries, tracing and multi-step orchestration, at the cost of hiding details you may need when debugging.

### What is the difference between tool use and function calling?

They describe the same mechanism. You give the model a list of functions with names, descriptions and parameter schemas; the model replies with a structured request naming a function and its arguments; your code executes it. Function calling is the API feature, and tool use is the broader term for the pattern.

### Are multi-agent systems better than a single agent?

Only for some problems. Splitting work among several specialised agents can help when subtasks are independent and need different tools or instructions. It also multiplies cost, latency and the ways a run can fail, and makes debugging harder. Start with one agent and split only when a measured problem justifies it.

### How do you test an AI agent?

Test at two levels. Unit-test the tools and the loop logic with a stand-in model that returns scripted tool calls. Then evaluate whole runs on a fixed set of tasks, checking the final outcome, the number of steps and whether any forbidden action was attempted. Recording every step makes failures diagnosable.