ClearCode Part 4: Autonomous Plan Execution, LLM-as-Judge, and Human-in-the-Loop Approval
Where Part 3 left us
Part 3 built four capabilities: short-term memory with session persistence, multi-step agent reasoning across tool calls, a three-tier progressive skills system, and MCP integration with GitHub and filesystem servers by default.
The agent could understand a codebase, remember a conversation, apply domain knowledge, and take individual actions via tools. But each action was still in direct response to a user prompt. One question, one answer. One instruction, one file edit.
Part 4 changes the interaction model. The agent can now plan.
The tasks layer
Six new files, one new command, one qualitative shift in what the agent can do.
clearcode/tasks/
├── planner.py # LLM to structured ExecutionPlan (Pydantic)
├── approval.py # Human-in-the-loop review: A/M/R loop with Rich table
├── task_store.py # SQLite WAL store with atomic state transitions
├── executor.py # Per-task agent with least-privilege tools + LLM-as-judge
├── orchestrator.py # Dependency loop, serial dispatch, retry, re-index after run
└── recovery.py # Resets IN_PROGRESS to PENDING on restart
The entry point is /plan . Everything else is invisible to the user until execution begins.
Step 1: Structured plan generation
planner.py calls the LLM with response_format=ExecutionPlan, which is OpenAI's structured output mode. The response is a Pydantic model, not free-form text.
ExecutionPlan contains a list of PlannedTask objects. Each task has:
id- a unique string identifier (e.g. task_001)title- a short description of what the task doestask_type- one of: design, implement, configure, review, test, integratedependencies- a list of task IDs that must be complete before this task can startacceptance_criteria- what the LLM judge will verify after execution
The task type is not cosmetic. executor.py uses it to select a least-privilege tool set per task. A design task does not get write access. A test task does not get the ability to create new files. Each task type gets exactly the tools it actually needs.
This is what a real generated plan looks like:
> /plan create a portfolio website for a backend engineer with 8 years of
experience using HTML, CSS, and JavaScript
Planning with LLM...
Plan: backend_engineer_portfolio
Goal: Professional portfolio site with dark theme, hero, skills, timeline,
projects, and contact section.
Stack: HTML · CSS · JavaScript | Est: 7.5h
ID Type Title Depends on
task_001 design Define site content architecture -
task_002 implement Create semantic HTML page structure task_001
task_003 configure Configure design tokens and dark theme task_002
task_004 implement Style hero, navigation, and header task_003
task_005 implement Style content sections task_003
task_006 implement Smooth scroll and active nav behavior task_002, task_004
task_007 implement Animations and scroll reveal task_005, task_006
task_008 implement Responsive layout refinements task_004, task_005
task_009 review Accessibility and SEO pass task_002, task_006
task_010 test Validate HTML and CSS quality task_008, task_009
task_011 test Test responsive behavior task_010
task_012 integrate Final integration and delivery check task_011
Twelve tasks. A typed dependency graph. Task types that map to different execution contexts. This is what the LLM returns as a structured Pydantic object before any execution begins.
Step 2: Human-in-the-loop approval
approval.py renders the plan as a Rich table and enters a loop:
[A]pprove / [M]odify task / [R]eject and re-plan:
Full /plan loop: goal input, plan generation, Rich table approval, serial execution, and re-index confirmation
A starts execution. M lets the user edit a specific task's fields before approving. R discards the plan and regenerates from the goal with optional guidance.
Nothing runs until the user approves. This is a design decision, not a missing feature. An autonomous agent that executes before the user has reviewed the plan is not something this architecture is built for.
The approval loop is synchronous by design. It uses rich.Prompt.ask() to block until the user responds, which means it blocks the async event loop during the wait. This is documented in CLAUDE.md as a known flaw. The correct fix is an async input primitive. For now, the blocking is predictable and bounded: it happens once, during approval, before any tool calls run.
Step 3: Persistence before execution
task_store.py writes the approved plan to SQLite before execution begins, not during and not after. WAL mode is enabled for atomic state transitions. Each task has four possible states: PENDING, IN_PROGRESS, COMPLETED, FAILED.
State transitions happen via SQL CASE expressions. The orchestrator never reads a partially updated state. If execution crashes mid-task, the store reflects the last committed state.
Config:
tasks:
db_path: .clearcode/tasks/tasks.db
db_path is CWD-relative, which means each project directory gets its own isolated task store.
Step 4: Execution with dependency resolution
orchestrator.py loops over the task graph and dispatches ready tasks: those whose dependencies are all in COMPLETED or SKIPPED state. Tasks execute serially. This is a deliberate tradeoff. Serial execution is predictable, debuggable, and avoids tool contention between concurrent agents. Parallel execution would require explicit locking on the filesystem and the index, neither of which is implemented yet.
For the portfolio plan above:
[A]pprove / [M]odify task / [R]eject and re-plan: A
Starting: [task_001] Define site content architecture
Completed: [task_001] Define site content architecture
Starting: [task_002] Create semantic HTML page structure
Completed: [task_002] Create semantic HTML page structure
...
Completed: [task_012] Final integration and delivery check
All 12 tasks completed successfully!
Re-indexing generated files so /ask can query them...
Index updated - you can now use /ask to ask about the generated code.
After the run, the orchestrator calls the indexer with force=True on any files created during execution. This bypasses the skip-if-exists guard and ensures that /ask queries work against the generated code immediately, without restarting.
Step 5: LLM-as-judge per task
executor.py builds a fresh agent for each task with the tool set appropriate for that task's task_type. After the agent completes the task, a second LLM call evaluates the output against the task's acceptance_criteria.
The judge returns a pass or fail verdict. On pass, the orchestrator marks the task COMPLETED and moves to the next ready task. On fail, fail_task increments the retry count and returns the task to PENDING for another attempt.
Tasks retry up to three times before being marked FAILED. A failed task does not stop the run. The orchestrator marks downstream dependents as SKIPPED and continues with tasks whose dependencies are otherwise satisfied.
Step 6: Crash recovery
recovery.py runs on every startup before any command is processed. It finds all tasks in IN_PROGRESS state and resets them to PENDING. This handles the case where the process crashed mid-execution: the next run picks up from the last completed task rather than leaving the store in an inconsistent state.
/task_status shows the current state of all tasks in the active project's store.
The honest flaws
These are all documented in CLAUDE.md.
Duplicate task IDs from the planner. The LLM occasionally generates two tasks with the same id (e.g. two tasks both named task_005). task_store.py does a direct INSERT with no deduplication guard. SQLite raises UNIQUE constraint failed after the user has already approved the plan and execution has started. The entire run is lost. The fix is to validate task IDs before the INSERT, not rely on the database constraint to surface the problem after approval.
Off-by-one in fail_task. fail_task increments retry_count unconditionally before checking whether retry_count >= max_retries. A task that exhausts all three retries ends up with retry_count = 4 in the DB, not 3. The retry log is misleading but the behavior is correct: the task is still marked FAILED after the third attempt.
get_dep_results omits output_files. When a task completes, its results (the text summary) are stored alongside a separate output_files field listing files it created. get_dep_results returns only id, title, and result. It omits output_files. A downstream agent that needs to read files written by its dependency has no way to find them. It sees only the text summary, not the actual generated artifacts. This is the most consequential flaw for any plan where one task writes files that another task must read and extend.
Async event loop blocking during approval. rich.Prompt.ask() is synchronous blocking I/O called inside async def _run_async. The event loop is blocked while waiting for user input at the approval step.
MCP subprocess leak. MultiServerMCPClient is not used as an async context manager. The MCP stdio subprocesses (npx processes for GitHub and filesystem servers) are started at agent creation time but never explicitly terminated. They run for the lifetime of the REPL process and are cleaned up only when the process exits.
What the demo shows
The GIF in the README shows the full /plan loop in real time: goal input, plan generation, Rich table display, approval, serial task execution with progress markers, and the re-index confirmation. The full terminal output is reproduced verbatim in the README.
Where Part 4 leaves things
All seven built layers are now wired together and running: context, agent, memory, MCP, skills, tasks, and the underlying LLM and embedder provider abstraction. The REPL has eight commands (/ask, /plan, /task_status, /show_index, /new_session, /switch, /session, /exit).
Three layers remain: safety, freshness, and eval. The eval layer is the one I am most deliberate about. It is the thing that makes all improvements measurable rather than impressionistic.
Part 5 is not yet decided. As always, drop a preference in the comments.
Full source: https://github.com/f2015537/clearcode
Part 1 - Architecture before code: https://blog.divyampatro.dev/clearcode-part-1-reverse-engineering-a-coding-agent-before-writing-a-single-line-of-code Part 2 - Context layer: https://blog.divyampatro.dev/clearcode-part-2-ast-aware-indexing-vector-stores-and-hybrid-retrieval Part 3 - Memory, MCP, and skills: https://blog.divyampatro.dev/clearcode-part-3-memory-agent-reasoning-skills-and-mcp