The problem
A complex task can be split among agents with roles: one plans, one writes code, one reviews it, one tests it. The code that decides who speaks next, what each one sees and when to stop (the orchestration) matters as much as the models. You will write it for four agents. Each is a function from its inputs to a string (in the tests, scripted fakes).
The protocol
plan = planner(task), once.- Then up to
max_roundsrounds, starting withfeedback = "":code = coder(task, plan, feedback);review = reviewer(code). If it does not start with"APPROVE", setfeedback = "review: " + reviewand start the next round: no test this round;- otherwise
result = tester(code). If it starts with"PASS", stop: success. Otherwise setfeedback = "test: " + resultand start the next round.
- Return
(code, log)on success, or(None, log)if the rounds run out.logis the list of(role, output)pairs in the order the agents spoke:("planner", plan),("coder", code),("reviewer", review),("tester", result).
Example. The coder writes
v1,v2,v3in turn; the reviewer answersREJECT: no error handling, thenAPPROVE, thenAPPROVE; the tester answersFAIL: test_empty, thenPASS.
- 1 — coder gets as feedback: (empty); reviewer:
REJECT: no error handling; tester: not run- 2 — coder gets as feedback:
review: REJECT: no error handling; reviewer:APPROVE; tester:FAIL: test_empty- 3 — coder gets as feedback:
test: FAIL: test_empty; reviewer:APPROVE; tester:PASSThe result is
("v3", log), where the log has 9 entries: planner, coder, reviewer, coder, reviewer, tester, coder, reviewer, tester.
What to implement
run_team(task, planner, coder, reviewer, tester, max_rounds): return (code, log), or (None, log).
Requirements
- The coder must receive the latest feedback, from the reviewer or the tester, prefixed as above.
- The tester only runs on approved code.
- Count rounds by calls to
coder: it is called at mostmax_roundstimes.