--- name: cli-process-loop-reliability category: software-development description: Diagnose and fix Hermes CLI process_loop failures that cause "AI stops receiving input after running for a while" --- # Hermes CLI process_loop Reliability ## Symptoms - CLI/TUI is still running and responsive to display - User can type in the input area - AI does not respond to any input - Only kill -9 terminates the process ## Root Cause The `process_loop` daemon thread in `cli.py` dies from an uncaught exception. The TUI main loop continues running, so the interface looks alive, but the queue consumer is dead — all user input goes into `_pending_input` and is never consumed. ## Critical Rules for process_loop ### Rule 1: Never let process_loop die ```python # MUST use BaseException, NOT Exception except BaseException as e: sys.stderr.write(f"[process_loop error] {type(e).__name__}: {e}\n") time.sleep(0.5) continue # Always continue the loop ``` ### Rule 2: Never use print() inside process_loop error handling - `print()` goes through `patch_stdout`'s `StdoutProxy` - StdoutProxy can fail during long sessions - A print() failure inside the exception handler kills process_loop permanently - Use `sys.stderr.write()` instead — it bypasses patch_stdout entirely ### Rule 3: Never block process_loop with .join() on worker threads - `_check_config_mcp_changes()` previously had `_reload_thread.join(timeout=30)` - This blocks process_loop for up to 30 seconds - All user input accumulates in the queue during this time - If the reload thread hangs, the TUI appears frozen - Fix: launch worker threads as daemon threads without joining ### Rule 4: State flags set before try blocks must be cleaned up on failure - `_voice_recording = True` was set before a try block - If `create_audio_recorder()` or config loading failed, the flag stayed True - Subsequent voice recording attempts would be silently ignored - Fix: wrap ALL code after the flag set in the same try block ## File Location `~/.hermes/hermes-agent/cli.py` Key functions: - `process_loop()` — input processing daemon thread (~line 10504) - `_check_config_mcp_changes()` — config watcher (~line 7177) - `_voice_start_recording()` — voice recording (~line 7460) ## Verification After making changes, verify: 1. No bare `except Exception` in process_loop — must be `except BaseException` 2. No `print()` in process_loop's error handler — must use `sys.stderr.write()` 3. No `.join()` calls inside `_check_config_mcp_changes()` — must be fire-and-forget 4. Any state flag set outside try block must have cleanup in the corresponding except