How to Stop AI From Being a Yes Man With an LLM Council
"You're absolutely right."
Shut up, Claude :)
I say that with love. But if you have used AI for more than 5 minutes, you know what I mean.
You can give it a half-baked product idea, 2 contradictory constraints, and just enough confidence to sound dangerous, and it still puts on the little customer success smile.
"Strong direction."
No. Sometimes the direction is a wall.
That is the annoying part about using LLMs for real decisions. Helpful often turns into agreeable. Agreeable is fine when you need a regex or first draft. It is bad when you are choosing architecture, product direction, pricing, security, or agent behavior that gets baked into the repo.
At that point, you need a few voices willing to make the idea uncomfortable before it becomes expensive.
That is where an LLM Council is useful.
My Spin On The LLM Council
Karpathy's LLM Council is the provider-based version: send one prompt to several LLMs, anonymize the answers, rank them, then synthesize the result.
My spin is different: instead of filling the council with providers, fill it with the personalities you wish the providers would reliably bring to the room.
The question I care about is less "which model is smartest?" and more:
"Will this thing tell me when my idea is bad?"
Most of the time, I do not want 5 versions of the same helpful answer. I want 5 jobs: Contrarian, First Principles Thinker, Expansionist, Outsider, and Executor.
You can run those roles on one model or several models. The important part is that each role gets the same brief independently and returns a focused critique.
One prompt that says "be skeptical, practical, strategic, simple, visionary, and concise" usually turns into soup. Parallel subagents keep the opinions separate long enough to disagree.
The 5 Voices I Want In The Room
I keep the role names plain because once you start giving them anime titles, the thing gets embarrassing fast.
| Role | What I want from it |
|---|---|
| Contrarian | Failure modes, weak assumptions, abuse cases, misleading claims |
| First Principles Thinker | The actual problem, minimum coherent workflow, missing proof |
| Expansionist | The stronger version of the idea, upside, adjacent capability |
| Outsider | What a normal person would find confusing, vague, or overcomplicated |
| Executor | The next action, verification check, owner, and defer list |
The point is to take what people notice when they compare models, that each one has a different temperament, and make those temperaments explicit.
Do not hope the model happens to be skeptical. Assign skepticism.
Do not ask one assistant to agree, disagree, expand, reduce, plan, and judge all at once. That is how you get a smooth paragraph that somehow avoids the main problem.
The council should be short:
1. Write one decision brief.
2. Spawn 5 role subagents in parallel.
3. Collect their findings.
4. Synthesize conflicts and agreements.
5. Choose one implementation move.
That is it. No tiny parliament living in your terminal. Just enough friction to stop AI from politely escorting a bad idea into production.
The Brief Does Most Of The Work
The brief matters more than the role names. If the brief is vague, 5 subagents just give you 5 flavors of fog. Very well formatted fog.
Use something like this:
Decision:
Context:
Constraints:
What success means:
What would make this decision wrong:
What should not be touched yet:
That last line is not decorative. AI agents love expanding scope. You ask whether a workflow should become a skill, and 4 minutes later it is proposing a platform and a new operating philosophy.
Relax. The council should make the next move clearer, not turn every decision into a migration plan with feelings.
What Changed When I Used It
I used this workflow while shaping this article.
A normal AI pass gave me sensible pros and cons. Very tidy. Very polite. Suspiciously "your idea is valid."
The council changed the shape of the work. The Contrarian caught that "5 subagents" could sound more autonomous than it really is. The First Principles Thinker dragged the article back to the real problem: AI agrees too easily. The Outsider flagged fake-sounding social proof. The Executor forced the setup instructions to become concrete.
The article stopped being "here is a clever AI council idea" and became "here is how to stop AI from yes-manning your idea before Codex or Claude Code builds."
That is the value. The council did not decide for me. It made the weak parts harder to ignore.
Add It To Codex
Codex supports skills for reusable workflows and subagents for parallel specialized work. So this is a good fit for a skill that spawns 5 focused reviewers before touching anything important.
I would keep it as a skill because the repeatable part is the method, not a new permanent persona. That is the split I argued for in Most Coding Agents Should Be Skills.
Create a personal skill:
mkdir -p ~/.codex/skills/decision-council
$EDITOR ~/.codex/skills/decision-council/SKILL.md
Use this as the starting point:
---
name: decision-council
description: Use when an AI assistant is likely to agree too quickly with a product, architecture, UX, security, or implementation decision.
---
Run 5 subagents in parallel:
1. Contrarian: find the flaw, weak assumption, abuse case, or misleading claim.
2. First Principles Thinker: restate the actual problem and minimum coherent workflow.
3. Expansionist: find the stronger version, upside, or adjacent capability.
4. Outsider: explain what is confusing to someone without project context.
5. Executor: define the next action, verification check, and defer list.
Return:
1. Council review
2. Strongest disagreement
3. Implementation move
4. Deferred risks
Do not treat this as a vote. Synthesize the strongest disagreement.
Then invoke it with a decision brief:
Use the decision-council skill.
Decision: Should we replace this hand-rolled workflow with a skill and 5 subagents?
Context: ...
Constraints: ...
What success means: ...
What would make this decision wrong: ...
What should not be touched yet: ...
This is the part people skip, then they blame the model. Do not type "review this idea." Give it the decision, constraints, and what would prove you wrong.
Add It To Claude Code
In Claude Code, define subagents and then create one skill or slash command that calls them. Same idea: make agreement harder to get by accident.
mkdir -p .claude/agents .claude/skills/decision-council
$EDITOR .claude/agents/contrarian.md
$EDITOR .claude/skills/decision-council/SKILL.md
Each subagent can be tiny:
---
name: contrarian
description: Reviews decisions for failure modes and weak assumptions.
tools: Read, Grep, Glob
model: inherit
---
Return the top 3 risks and the assumption most likely to be false.
Repeat that shape for first-principles-thinker, expansionist, outsider, and executor. Then make the skill:
---
description: Run a 5-role decision council using parallel subagents.
disable-model-invocation: true
---
Decision brief:
$ARGUMENTS
Run these subagents in parallel:
- contrarian
- first-principles-thinker
- expansionist
- outsider
- executor
Return consensus, strongest disagreement, implementation move, and deferred risks.
Now the command becomes:
/decision-council "Decision: ... Context: ... Constraints: ..."
If you use Claude Code or Codex often, this is worth turning into a habit. Most decisions do not need ceremony. The few that matter deserve more than "great point."
When Not To Use It
Do not run a council for tiny choices. Naming a variable does not need 5 AI agents and a committee chair. Please just name the variable and move on with your life.
Do not use it when the real bottleneck is missing data. If nobody knows the customer requirement, 5 role prompts will only produce cleaner speculation. It is still guessing in a nicer outfit.
Use it when the cost of being wrong is real: architecture, security, product direction, pricing, data model changes, public claims, and agent behavior other people will depend on.
The goal is not consensus.
The goal is a better next move.
Sources: Karpathy's LLM Council repo, research on multiagent debate, OpenAI Codex subagents docs, OpenAI Codex skills docs, Claude Code subagents docs, and Claude Code skills docs.
Tagged
