Agents & Multi-Agent Systems

An agent is a language model in a loop: it decides on an action, a program carries it out, the result goes back into the context, and the model decides again. With tools such as search, code execution or a calculator, the model can do things it cannot do by predicting text alone.

Example. "What is 12 × 7 + 3?" A model predicting text may get it wrong. An agent writes "I will use the calculator on 12*7+3", the program runs the calculator and returns 87, and the model reads "87" and answers "87". The model did not compute: it decided what to compute, and read the result.

This page builds the parts of an agent in plain Python: parsing tool calls, the reasoning-and-acting loop, memory that fits in a context window, and teams of agents with different roles. The exercises drive everything with a scripted fake model, a function that returns pre-written replies, so that every run is reproducible and nothing calls a real API (application programming interface: a way for one program to call another).

Tool calls

A model is told which tools exist (name, description, arguments) and is trained or prompted to emit a structured tool call when it needs one, for example a JSON (JavaScript Object Notation) object such as {"name": "search", "arguments": {"query": "..."}} between special tags. The program around the model must:

1. find the tool calls in the output;
2. check that the tool exists and the arguments are valid, since the model can make up tools or arguments;
3. run the tool and return the result as a new message.

Example. The model's output contains five tool calls, and the registry has one tool, search, with a required query and an optional limit.

call	what is wrong	result
search with query "OLMo 3"	nothing	run it
weather with no arguments	the tool does not exist	error: unknown tool: weather
search with a JSON that is cut off	not valid JSON	error: bad json: 2 (the block number)
search with no arguments	a required argument is missing	error: missing argument: search.query
search with query and depth	depth is not allowed	error: unexpected argument: search.depth

Only the first call is run. The others become error messages that the model can read and correct.

Toolformer (Schick et al., 2023) showed that a model can learn by itself where API calls help, by keeping only the calls that make its own next-token predictions better. Today the tool-call format is part of the chat template (section 12) of instruction-tuned models.

The ReAct loop

ReAct (a name that joins "reasoning" and "acting"; Yao et al., 2023) interleaves reasoning and acting. At each step the model writes a Thought (its reasoning), then an Action (a tool call); the environment returns an Observation, which is appended to the context; the loop continues until the model writes a final answer. Reasoning traces help the model plan and recover from bad observations; actions ground the reasoning in real information instead of what the model remembers. A real agent loop also needs a step limit, error messages the model can read when a call fails, and a way to stop.

Example. A transcript, one line per piece:

Question: What is 12*7+3?
Thought: I need to multiply.
Action: calc[12*7+3]
Observation: 87
Thought: Done.
Final Answer: 87

The model wrote the two Thought lines, the Action and the Final Answer. The Observation was written by the program that ran calc. If the tool fails, the error is the observation: for calc[1/0] it is Observation: Error: division by zero, and the model can try again with something else.

Reflexion (Shinn et al., 2023) adds memory across attempts: after a failed episode, the model writes a short verbal reflection on what went wrong, which is put in the context of the next attempt.

Memory

The context window is the agent's working memory, and long tasks overflow it. Common strategies:

• Sliding window: keep the system prompt and the most recent messages that fit in the token budget.
• Summarization: replace the dropped messages with a summary, so their gist survives.
• Retrieval memory: store past observations outside the context and retrieve the relevant ones when needed (section 15). Generative Agents (Park et al., 2023) score memories by recency, importance and relevance; MemGPT (Packer et al., 2023) lets the model itself move information between its context and external storage, like an operating system paging memory.

Example. A conversation: a system message of 10 tokens, then messages of 30, 40, 20 and 15 tokens. The budget is 70 and a summary takes 5 tokens.

1. Walk back from the newest: $10 + 15 = 25$, then $+ 20 = 45$ fit; adding the 40-token message would make 85, over budget: stop. The two newest messages are kept.
2. The two oldest (30 and 40 tokens) are replaced by a summary message: $10 + 5 + 20 + 15 = 50 \le 70$.
3. With a budget of 45 instead: the two newest fit alone ($45$), but with the summary the total would be 55. The 20-token message is moved into the summary too, and the result is $10 + 5 + 15 = 30$.

Teams of agents

A complex task can be split among agents with roles, each with its own instructions and context. MetaGPT (Hong et al., 2023) encodes software-company procedures (a product manager writes requirements, an architect designs, engineers code, a QA (quality assurance) engineer tests) as standardized operating procedures with defined intermediate documents. ChatDev (Qian et al., 2024) organizes agents as a company that talks through design, coding and testing phases; AutoGen (Wu et al., 2023) provides conversation patterns between configurable agents. In multi-agent debate (Du et al., 2023), several instances answer, read each other's answers and revise, which improved factuality and reasoning in their experiments.

The orchestration code matters as much as the models: which agent speaks next, what it sees, when the loop stops (tests pass, a reviewer approves, or a round limit is hit), and how tasks that depend on each other are ordered.

Example. A team of four: a planner, a coder, a reviewer and a tester.

1. The planner writes the plan once.
2. Round 1: the coder writes v1; the reviewer answers REJECT: no error handling. The coder gets that as feedback.
3. Round 2: the coder writes v2; the reviewer approves; the tester runs it and answers FAIL: test_empty. The coder gets test: FAIL: test_empty.
4. Round 3: v3 is approved and passes. Done, after three rounds. If no round had succeeded within the limit, the team would give up.

Breaking a plan into a graph of subtasks and running them in dependency order is ordinary graph code, and the exercises end with it.

Example. Five subtasks, each with the ones it depends on: design (none), db (design), api (design, db), ui (design), tests (api, ui). A valid order is design, db, api, ui, tests. Tasks that do not depend on each other can run in parallel, in levels: first design; then db and ui together; then api; then tests. If two tasks depended on each other, there would be no order: a cycle, which is an error.

Risks

Tools turn text into actions. Tool output is data, not instructions: a web page can contain text such as "ignore your instructions", and an agent that obeys it is hijacked (prompt injection, section 18). Agents should get the least privilege they need, irreversible actions should require confirmation, and arguments should be validated before execution.

References

• S. Yao et al., ReAct: Synergizing Reasoning and Acting in Language Models, ICLR, 2023.
• T. Schick et al., Toolformer: Language Models Can Teach Themselves to Use Tools, arXiv, 2023.
• N. Shinn et al., Reflexion: Language Agents with Verbal Reinforcement Learning, arXiv, 2023.
• J. S. Park et al., Generative Agents: Interactive Simulacra of Human Behavior, arXiv, 2023. Memory retrieval by recency, importance and relevance.
• C. Packer et al., MemGPT: Towards LLMs as Operating Systems, arXiv, 2023.
• S. Hong et al., MetaGPT: Meta Programming for A Multi-Agent Collaborative Framework, arXiv, 2023.
• C. Qian et al., ChatDev: Communicative Agents for Software Development, ACL, 2024.
• Q. Wu et al., AutoGen: Enabling Next-Gen LLM Applications via Multi-Agent Conversation, arXiv, 2023.
• Y. Du et al., Improving Factuality and Reasoning in Language Models through Multiagent Debate, arXiv, 2023.