AI agents explained: tools, loops and when not to use one

An AI agent is an LLM application where the model, not the programmer, decides the next step. The model is given a goal and a set of tools, chooses which tool to call, reads the result and repeats until it judges the task complete. That is the whole idea behind AI agents explained without the marketing: a model inside a loop, with tools. The loop gives flexibility on open-ended tasks and takes away predictability, which is why knowing when not to use an agent matters as much as knowing how to build one.

Agents come late in the AI engineer roadmap because they depend on LLM APIs, prompting and retrieval. If retrieval is new to you, read how RAG works first.

What is an AI agent?

An AI agent is a system made of three parts: a language model that reasons about what to do, a set of tools the model may ask to use, and a control loop that runs those tools and feeds the results back. The defining feature is that the sequence of steps is chosen at run time by the model.

Compare two ways of handling a refund request. In a fixed program, the code always looks up the order, checks the policy and then drafts a reply. In an agent, the model is told it has an order lookup tool, a policy search tool and a refund tool, and it works out which to call and in what order for each case, including the unusual ones nobody planned for.

How does tool use work?

Tool use works by the model producing a structured request that your program then executes; the model itself never runs anything. You describe each tool to the model with a name, a plain-language description and a schema for its arguments. When the model decides a tool would help, it returns the tool's name and arguments instead of a text answer. Your code validates the request, runs the function and sends the result back as the next message.

Two consequences follow. First, tool descriptions are prompts: a vague description produces wrong calls, and two tools with overlapping purposes confuse the model. Second, your code sits between the model and every action, which is where all real control lives. The model can ask to delete a record; whether that happens is your program's decision.

How does the agent loop work?

The agent loop works by alternating model calls and tool executions until the model returns a final answer or a limit stops it.

A minimal version of that loop is short. In the sketch below, call_model stands for whichever model API you use and returns a reply that either contains text or a tool request.

def run_agent(task: str, tools: dict, call_model, max_steps: int = 8) -> str:
    messages = [{"role": "user", "content": task}]
    for _ in range(max_steps):
        reply = call_model(messages, tools)
        if reply.tool_call is None:
            return reply.text  # the model has finished
        name, args = reply.tool_call.name, reply.tool_call.args
        if name not in tools:
            result = f"error: unknown tool {name}"
        else:
            try:
                result = tools[name](**args)
            except Exception as exc:
                result = f"error: {exc}"
        messages.append({"role": "assistant", "tool_call": reply.tool_call})
        messages.append({"role": "tool", "name": name, "content": str(result)})
    return "stopped: step limit reached"

Notice what the sketch does with failures. An unknown tool or a raised exception becomes a message back to the model, which can then correct itself, and the step limit guarantees the loop ends. Planning, memory and reflection, the terms you will meet in agent literature, are refinements of this loop: asking the model to write a plan first, storing notes between steps, or having it review its own work before finishing.

What is the difference between an agent and a workflow?

The difference is who decides the sequence of steps. In a workflow, the developer fixes the path in code and the model fills in individual steps, such as classifying a ticket or drafting a reply. In an agent, the model chooses the path. Many production systems described as agents are workflows with one or two model-driven decisions, and that is usually a sound design.

When should you not use an agent?

Do not use an agent when you can write down the steps beforehand. If the task can be drawn as a flowchart, a workflow will be cheaper, faster, easier to test and easier to explain to a risk team. Reach for an agent only when the path really does depend on intermediate results that cannot be anticipated.

The underlying reason is that errors compound. Each step has some chance of going wrong, and a run succeeds only if enough steps go right, so a long chain is noticeably less reliable than any one of its links. Every additional step also adds a model call, which adds cost and delay.

What guardrails does an agent need?

An agent needs guardrails enforced in code, because instructions in a prompt are requests and not controls. The essential ones are:

• Limits. A maximum number of steps, a time limit and a spending limit per run, so a confused agent cannot loop indefinitely.
• Least privilege. Give each agent only the tools it needs, with the narrowest permissions. Prefer read-only tools; scope write tools to the current user's own data.
• Argument validation. Check every tool call against its schema and against business rules before running it.
• Human approval. Require confirmation for actions that are irreversible or costly: payments, deletions, messages sent to customers.
• Untrusted input handling. Text returned by tools, such as web pages, emails and retrieved documents, may contain instructions written to hijack the agent. This is prompt injection. Treat tool output as data, and never let it expand what the agent is allowed to do.
• Tracing. Record every model call, tool call and result. Without a trace, a failed run cannot be diagnosed.

What do interviewers ask about agents?

Interviewers ask about agents to find out whether you have judgement, not whether you know a framework. Common prompts are to design an agent for a concrete task, to explain how you would stop it doing damage, and to say how you would measure whether it works. Expect a follow-up asking why the task needs an agent at all; a candidate who argues for a simpler workflow where one fits usually scores higher than one who reaches for autonomy by default.

Measuring an agent means scoring whole tasks on a fixed test set, covered in how to evaluate an LLM application. Typical questions with answer structures are in GenAI interview questions. Agents and tool use are core skills for the GenAI and LLM engineer role and the AI backend engineer role; you can compare what each track expects in the role explorer.