Definition
What is an LLM Council?
An LLM council is a structured way to ask several language models the same question, let them work independently, have them review one another, and then return one final answer with consensus and dissent visible.
The short answer
An LLM council is not just a list of model outputs. The useful part is the process: independent answers first, anonymous review second, synthesis last.
That structure makes the answer less dependent on one model's blind spots. When models disagree, the disagreement becomes useful evidence instead of being hidden.
- Several models answer the same task independently
- The answers are reviewed and ranked before synthesis
- The final response keeps the main verdict and the strongest dissent
Why people search for it
Users want better reliability than one chatbot can provide. A council helps with complex work where a plausible answer is not enough: strategy, research, contracts, pricing, product decisions, and document-heavy analysis.
Where the idea comes from
The name comes from Andrej Karpathy's open-source llm-council project (late 2025): several models answer, anonymously grade each other, and a chairman synthesizes. He described it as an unsupported weekend hack, and the idea spread far beyond it — by February 2026, Perplexity had shipped a Model Council feature on its Max plan.
The method itself is older than the name. Peer consensus among independent evaluators was showing measurable gains in peer-reviewed research as far back as 2013 — deliberation beats single judgment when the judges are independent and the disagreement is kept visible. LLM Council is built on that lineage.
What production data shows
Councils generate evidence about the models themselves. Across 237,177 answers judged in peer review on this platform, one pattern stands out: models favor their own work. In 73,580 paired judgments, 47 of 49 models ranked their own answer higher than peers ranked the identical answer — the strongest frontier models most of all.
That is why the judging design matters more than the model count. Anonymous cross-model review makes self-preference measurable; a single synthesizer model as judge inherits it silently. Read the full study at https://llmcouncil.ai/ai-self-preference-bias-study.
How LLM Council uses it
LLM Council turns the idea into a product workflow. It identifies the intent, builds the context, seats the right model pool, runs council stages, and produces one answer or document artifact.
Free users can try the workflow daily. Pro adds stronger model pools, longer context, exports, and higher effort. Fox adds the strongest frontier model pool, highest context, and maximum council intelligence.
When it is worth using
Use an LLM council when the cost of a wrong answer is higher than the cost of waiting for peer review.
- A decision with tradeoffs or dissent
- A document that needs review before sending
- A research answer that needs current context and citations
- A prompt where one model keeps missing the user's intent