pipeline_core/skills_library/all/cli-process-loop-reliability/SKILL.md

name	category	description
cli-process-loop-reliability	software-development	Diagnose and fix Hermes CLI process_loop failures that cause "AI stops receiving input after running for a while"

Hermes CLI process_loop Reliability

Symptoms

• CLI/TUI is still running and responsive to display
• User can type in the input area
• AI does not respond to any input
• Only kill -9 terminates the process

Root Cause

The process_loop daemon thread in cli.py dies from an uncaught exception. The TUI main loop continues running, so the interface looks alive, but the queue consumer is dead — all user input goes into _pending_input and is never consumed.

Critical Rules for process_loop

Rule 1: Never let process_loop die

# MUST use BaseException, NOT Exception
except BaseException as e:
    sys.stderr.write(f"[process_loop error] {type(e).__name__}: {e}\n")
    time.sleep(0.5)
    continue  # Always continue the loop

Rule 2: Never use print() inside process_loop error handling

• print() goes through patch_stdout's StdoutProxy
• StdoutProxy can fail during long sessions
• A print() failure inside the exception handler kills process_loop permanently
• Use sys.stderr.write() instead — it bypasses patch_stdout entirely

Rule 3: Never block process_loop with .join() on worker threads

• _check_config_mcp_changes() previously had _reload_thread.join(timeout=30)
• This blocks process_loop for up to 30 seconds
• All user input accumulates in the queue during this time
• If the reload thread hangs, the TUI appears frozen
• Fix: launch worker threads as daemon threads without joining

Rule 4: State flags set before try blocks must be cleaned up on failure

• _voice_recording = True was set before a try block
• If create_audio_recorder() or config loading failed, the flag stayed True
• Subsequent voice recording attempts would be silently ignored
• Fix: wrap ALL code after the flag set in the same try block

File Location

~/.hermes/hermes-agent/cli.py

Key functions:

• process_loop() — input processing daemon thread (~line 10504)
• _check_config_mcp_changes() — config watcher (~line 7177)
• _voice_start_recording() — voice recording (~line 7460)

Verification

After making changes, verify:

1. No bare except Exception in process_loop — must be except BaseException
2. No print() in process_loop's error handler — must use sys.stderr.write()
3. No .join() calls inside _check_config_mcp_changes() — must be fire-and-forget
4. Any state flag set outside try block must have cleanup in the corresponding except