# ClearCode Part 4: Autonomous Plan Execution, LLM-as-Judge, and Human-in-the-Loop Approval ## Where Part 3 left us Part 3 built four capabilities: short-term memory with session persistence, multi-step agent reasoning across tool calls, a three-tier progressive skills system, and MCP integration with GitHub and filesystem servers by default. The agent could understand a codebase, remember a conversation, apply domain knowledge, and take individual actions via tools. But each action was still in direct response to a user prompt. One question, one answer. One instruction, one file edit. Part 4 changes the interaction model. The agent can now plan. * * * ## The tasks layer Six new files, one new command, one qualitative shift in what the agent can do. ```plaintext clearcode/tasks/ ├── planner.py # LLM to structured ExecutionPlan (Pydantic) ├── approval.py # Human-in-the-loop review: A/M/R loop with Rich table ├── task_store.py # SQLite WAL store with atomic state transitions ├── executor.py # Per-task agent with least-privilege tools + LLM-as-judge ├── orchestrator.py # Dependency loop, serial dispatch, retry, re-index after run └── recovery.py # Resets IN_PROGRESS to PENDING on restart ``` The entry point is `/plan` . Everything else is invisible to the user until execution begins. * * * ## Step 1: Structured plan generation `planner.py` calls the LLM with `response_format=ExecutionPlan`, which is OpenAI's structured output mode. The response is a Pydantic model, not free-form text. `ExecutionPlan` contains a list of `PlannedTask` objects. Each task has: * `id` - a unique string identifier (e.g. task\_001) * `title` - a short description of what the task does * `task_type` - one of: design, implement, configure, review, test, integrate * `dependencies` - a list of task IDs that must be complete before this task can start * `acceptance_criteria` - what the LLM judge will verify after execution The task type is not cosmetic. `executor.py` uses it to select a least-privilege tool set per task. A design task does not get write access. A test task does not get the ability to create new files. Each task type gets exactly the tools it actually needs. This is what a real generated plan looks like: ```plaintext > /plan create a portfolio website for a backend engineer with 8 years of experience using HTML, CSS, and JavaScript Planning with LLM... Plan: backend_engineer_portfolio Goal: Professional portfolio site with dark theme, hero, skills, timeline, projects, and contact section. Stack: HTML · CSS · JavaScript | Est: 7.5h ID Type Title Depends on task_001 design Define site content architecture - task_002 implement Create semantic HTML page structure task_001 task_003 configure Configure design tokens and dark theme task_002 task_004 implement Style hero, navigation, and header task_003 task_005 implement Style content sections task_003 task_006 implement Smooth scroll and active nav behavior task_002, task_004 task_007 implement Animations and scroll reveal task_005, task_006 task_008 implement Responsive layout refinements task_004, task_005 task_009 review Accessibility and SEO pass task_002, task_006 task_010 test Validate HTML and CSS quality task_008, task_009 task_011 test Test responsive behavior task_010 task_012 integrate Final integration and delivery check task_011 ``` Twelve tasks. A typed dependency graph. Task types that map to different execution contexts. This is what the LLM returns as a structured Pydantic object before any execution begins. * * * ## Step 2: Human-in-the-loop approval `approval.py` renders the plan as a Rich table and enters a loop: ```plaintext [A]pprove / [M]odify task / [R]eject and re-plan: ``` ![ClearCode /plan demo](https://raw.githubusercontent.com/f2015537/clearcode/main/demo/clearcode-plan-opt.gif align="center") *Full /plan loop: goal input, plan generation, Rich table approval, serial execution, and re-index confirmation* A starts execution. M lets the user edit a specific task's fields before approving. R discards the plan and regenerates from the goal with optional guidance. Nothing runs until the user approves. This is a design decision, not a missing feature. An autonomous agent that executes before the user has reviewed the plan is not something this architecture is built for. The approval loop is synchronous by design. It uses `rich.Prompt.ask()` to block until the user responds, which means it blocks the async event loop during the wait. This is documented in CLAUDE.md as a known flaw. The correct fix is an async input primitive. For now, the blocking is predictable and bounded: it happens once, during approval, before any tool calls run. * * * ## Step 3: Persistence before execution `task_store.py` writes the approved plan to SQLite before execution begins, not during and not after. WAL mode is enabled for atomic state transitions. Each task has four possible states: PENDING, IN\_PROGRESS, COMPLETED, FAILED. State transitions happen via SQL CASE expressions. The orchestrator never reads a partially updated state. If execution crashes mid-task, the store reflects the last committed state. Config: ```yaml tasks: db_path: .clearcode/tasks/tasks.db ``` `db_path` is CWD-relative, which means each project directory gets its own isolated task store. * * * ## Step 4: Execution with dependency resolution `orchestrator.py` loops over the task graph and dispatches ready tasks: those whose dependencies are all in COMPLETED or SKIPPED state. Tasks execute serially. This is a deliberate tradeoff. Serial execution is predictable, debuggable, and avoids tool contention between concurrent agents. Parallel execution would require explicit locking on the filesystem and the index, neither of which is implemented yet. For the portfolio plan above: ```plaintext [A]pprove / [M]odify task / [R]eject and re-plan: A Starting: [task_001] Define site content architecture Completed: [task_001] Define site content architecture Starting: [task_002] Create semantic HTML page structure Completed: [task_002] Create semantic HTML page structure ... Completed: [task_012] Final integration and delivery check All 12 tasks completed successfully! Re-indexing generated files so /ask can query them... Index updated - you can now use /ask to ask about the generated code. ``` After the run, the orchestrator calls the indexer with `force=True` on any files created during execution. This bypasses the skip-if-exists guard and ensures that `/ask` queries work against the generated code immediately, without restarting. * * * ## Step 5: LLM-as-judge per task `executor.py` builds a fresh agent for each task with the tool set appropriate for that task's `task_type`. After the agent completes the task, a second LLM call evaluates the output against the task's `acceptance_criteria`. The judge returns a pass or fail verdict. On pass, the orchestrator marks the task COMPLETED and moves to the next ready task. On fail, `fail_task` increments the retry count and returns the task to PENDING for another attempt. Tasks retry up to three times before being marked FAILED. A failed task does not stop the run. The orchestrator marks downstream dependents as SKIPPED and continues with tasks whose dependencies are otherwise satisfied. * * * ## Step 6: Crash recovery `recovery.py` runs on every startup before any command is processed. It finds all tasks in IN\_PROGRESS state and resets them to PENDING. This handles the case where the process crashed mid-execution: the next run picks up from the last completed task rather than leaving the store in an inconsistent state. `/task_status` shows the current state of all tasks in the active project's store. * * * ## The honest flaws These are all documented in CLAUDE.md. **Duplicate task IDs from the planner.** The LLM occasionally generates two tasks with the same id (e.g. two tasks both named task\_005). `task_store.py` does a direct INSERT with no deduplication guard. SQLite raises UNIQUE constraint failed after the user has already approved the plan and execution has started. The entire run is lost. The fix is to validate task IDs before the INSERT, not rely on the database constraint to surface the problem after approval. **Off-by-one in fail\_task.** `fail_task` increments `retry_count` unconditionally before checking whether `retry_count >= max_retries`. A task that exhausts all three retries ends up with `retry_count = 4` in the DB, not `3`. The retry log is misleading but the behavior is correct: the task is still marked FAILED after the third attempt. **get\_dep\_results omits output\_files.** When a task completes, its results (the text summary) are stored alongside a separate `output_files` field listing files it created. `get_dep_results` returns only `id`, `title`, and `result`. It omits `output_files`. A downstream agent that needs to read files written by its dependency has no way to find them. It sees only the text summary, not the actual generated artifacts. This is the most consequential flaw for any plan where one task writes files that another task must read and extend. **Async event loop blocking during approval.** `rich.Prompt.ask()` is synchronous blocking I/O called inside `async def _run_async`. The event loop is blocked while waiting for user input at the approval step. **MCP subprocess leak.** `MultiServerMCPClient` is not used as an async context manager. The MCP stdio subprocesses (npx processes for GitHub and filesystem servers) are started at agent creation time but never explicitly terminated. They run for the lifetime of the REPL process and are cleaned up only when the process exits. * * * ## What the demo shows The GIF in the README shows the full `/plan` loop in real time: goal input, plan generation, Rich table display, approval, serial task execution with progress markers, and the re-index confirmation. The full terminal output is reproduced verbatim in the README. * * * ## Where Part 4 leaves things All seven built layers are now wired together and running: context, agent, memory, MCP, skills, tasks, and the underlying LLM and embedder provider abstraction. The REPL has eight commands (/ask, /plan, /task\_status, /show\_index, /new\_session, /switch, /session, /exit). Three layers remain: safety, freshness, and eval. The eval layer is the one I am most deliberate about. It is the thing that makes all improvements measurable rather than impressionistic. Part 5 is not yet decided. As always, drop a preference in the comments. Full source: https://github.com/f2015537/clearcode Part 1 - Architecture before code: https://blog.divyampatro.dev/clearcode-part-1-reverse-engineering-a-coding-agent-before-writing-a-single-line-of-code Part 2 - Context layer: https://blog.divyampatro.dev/clearcode-part-2-ast-aware-indexing-vector-stores-and-hybrid-retrieval Part 3 - Memory, MCP, and skills: https://blog.divyampatro.dev/clearcode-part-3-memory-agent-reasoning-skills-and-mcp