The concept
What is an LLM Council?
An LLM Council is a group of AI models that answer the same question independently, review each other anonymously, and produce one final answer. Single models make confident mistakes. Models are far better at catching each other than at catching themselves.
The definition
One task goes before several frontier models at once. Three stages, always in this order:
- Independent answers — every model answers alone, without seeing the others
- Anonymous peer review — every model ranks the answers without knowing which model wrote which
- Synthesis — a chairman model folds the strongest answers and the strongest dissent into one verdict
Why it works
Independence prevents anchoring: a model that has seen another answer drifts toward it. Anonymity prevents brand bias: judges rate the answer, not the author. Peer review catches what self-review cannot — a model rarely flags its own blind spots, while other models flag them readily.
The result is an answer that has survived scrutiny before anyone acts on it.
Where the idea comes from
Group deliberation beating individual judgment is older than AI. Our founder published peer-reviewed research on structured group consensus at Arizona State University in 2013: peer networks outperform single-source judgment by a wide margin.
In late 2024, Andrej Karpathy popularised the term with a small open-source experiment that put several models in council. LLM Council is the productised form of the idea — the same method, run at production scale, with every judgment on the record.
A council is not an ensemble
Classic ensembles average outputs. A council argues. Averaging hides disagreement, which is exactly the information a serious decision needs. A council keeps dissent visible all the way into the final verdict.
See a council run
LLM Council runs the method as a product: bring a document or a question, and multiple frontier models answer, review, and return one verdict. The live count of answers judged in peer review is on the homepage at https://llmcouncil.ai — real production numbers, not rounded up.