pipeline_core/skills_library/all/hermes-state-merge/SKILL.md

name	description	version	metadata
hermes-state-merge	Merge skills and memory between two Hermes instances using a shared backup repository, without overwriting existing state.	1.0.0	hermes tags backup merge sync state skills memory migration

Hermes State Merge

Incrementally merge skills, memory, and config between Hermes instances sharing a common backup repository (e.g., https://git.opencomputing.cn/yumoqing/hermes-backup). Unlike full restore which overwrites, merge only adds what's missing and reconciles differences.

When to Use

• Two Hermes instances (A and B) both push to the same backup repo
• Instance B wants to get A's new skills/memory without losing its own
• Periodic synchronization between instances

What Gets Merged

Component	Strategy	Conflict handling
Skills	Incremental file copy	--ignore-existing for new, manual review for modified
Memory	Read from backup DB, write via memory tool	Skip duplicates by content hash
Config	YAML merge with precedence rules	Local config wins unless explicitly overridden
Cron jobs	Add missing jobs only	Skip if job_id exists

Workflow

Phase 1: Clone Backup to Temp Directory

CLONE_DIR=$(mktemp -d)
git clone --depth 1 git@<host>:<user>/<repo>.git "$CLONE_DIR"

If HTTPS with token:

git clone --depth 1 https://<token>@<host>/<user>/<repo>.git "$CLONE_DIR"

Phase 2: Skills Merge

# 1. List skills in both directories
BACKUP_SKILLS="$CLONE_DIR/skills"
LOCAL_SKILLS="$HOME/.hermes/skills"

# 2. Copy new skills (don't overwrite existing)
rsync -av --ignore-existing "$BACKUP_SKILLS/" "$LOCAL_SKILLS/" 2>/dev/null

# 3. Identify modified skills (different content)
diff -rq "$BACKUP_SKILLS" "$LOCAL_SKILLS" 2>/dev/null | grep "differ"

# 4. For each modified skill, compare and decide:
#    - If backup version has new content not in local → merge
#    - If local version has fixes not in backup → keep local
#    - If both changed significantly → keep backup version (newer source of truth)

Modified skill merge pattern:

# For each modified skill, use diff to decide
diff -u "$LOCAL_SKILLS/$skill/SKILL.md" "$BACKUP_SKILLS/$skill/SKILL.md"
# If backup has significant additions → cp from backup
# If local has fixes → keep local

Phase 3: Memory Merge

Memory is stored in SQLite (memory_store.db). Cannot directly copy — must merge by content.

# Read memory entries from backup DB
import sqlite3, json

backup_db = sqlite3.connect(f"{CLONE_DIR}/memory_store.db")
local_db = sqlite3.connect(f"{HOME}/.hermes/memory_store.db")

# Extract entries from backup
backup_entries = backup_db.execute("SELECT * FROM memory").fetchall()
local_entries = local_db.execute("SELECT * FROM memory").fetchall()

local_set = {row[1] for row in local_entries}  # assuming content is index 1

for entry in backup_entries:
    content = entry[1]
    if content not in local_set:
        # This entry doesn't exist locally — add it
        local_db.execute("INSERT INTO memory VALUES (?, ?, ?)", entry)
        local_db.commit()

backup_db.close()
local_db.close()

Alternative (via Hermes memory tool): For each new memory entry found in backup, call memory(action='add', target='memory', content='...') in a Hermes session.

Phase 4: Config Merge

import yaml

def deep_merge(base, override, overwrite=False):
    """Merge override into base. If overwrite=False, base wins on conflicts."""
    for key, val in override.items():
        if key not in base:
            base[key] = val
        elif isinstance(val, dict) and isinstance(base.get(key), dict):
            deep_merge(base[key], val, overwrite)
        elif overwrite:
            base[key] = val
    return base

with open(f"{CLONE_DIR}/config.yaml") as f:
    backup_config = yaml.safe_load(f)
with open(f"{HOME}/.hermes/config.yaml") as f:
    local_config = yaml.safe_load(f)

# Merge: local wins on conflicts (preserve local API keys, etc.)
merged = deep_merge(backup_config, local_config, overwrite=False)

# Write merged config
with open(f"{HOME}/.hermes/config.yaml", 'w') as f:
    yaml.dump(merged, f, default_flow_style=False)

Phase 5: Cleanup

rm -rf "$CLONE_DIR"

Related Skills

• hermes-instance-migration — Full rsync-based migration to a new machine (includes venv rebuild + multi-user setup). Use that when moving to a new server; use this skill for periodic state sync between running instances.
• hermes-state-backup — Git-based backup to remote repo.

Pitfalls

1. Never merge state.db directly — it contains session state and can cause corruption if merged between instances. Let each instance maintain its own state.
2. Never include state.db in git backups — it is a binary SQLite file (~1-2GB per snapshot) that git cannot delta-compress. Over 100+ commits this bloats the repo to tens of GB (observed: 176 commits × 1.4GB = 81GB pack). The backup script must exclude state.db, state.db-shm, state.db-wal. Each instance regenerates its own state.db from sessions.
3. Backup script must clean dirty working tree before checkout — add git reset --hard HEAD and git clean -fdx -e .git after fetch and before git checkout -B. Without this, any uncommitted changes (e.g. from a prior failed backup or concurrent cron output) will abort the checkout and fail the entire backup silently.
4. auth.json should never be merged — OAuth tokens are instance-specific. Each instance needs its own authentication.
5. Skills with same name but different content — compare carefully. A skill modified locally may have instance-specific fixes.
6. Memory entries are plain text — use content-based dedup, not ID-based, since IDs may differ between instances.
7. Config merge should NOT overwrite API keys — always let local config win on sensitive fields.
8. memory_store.db schema may differ — if the backup is from a significantly older Hermes version, the schema may differ. Check schema first.
9. Cron job conflicts — cron jobs have unique IDs. Don't merge cron jobs; instead, compare schedules and prompts manually.
10. Large skill repos — some skill sets include templates, schemas, assets. rsync --ignore-existing is fast for initial merge, but diff -rq for subsequent comparisons may be slow with hundreds of skills.

Automation Script

Save as ~/.hermes/scripts/merge-from-backup.sh:

#!/bin/bash
set -e

REMOTE="git@<host>:<user>/<repo>.git"
CLONE_DIR=$(mktemp -d)
LOCAL_HERMES="$HOME/.hermes"

echo "Cloning backup repo..."
git clone --depth 1 "$REMOTE" "$CLONE_DIR"

echo "Merging skills (incremental)..."
rsync -av --ignore-existing "$CLONE_DIR/skills/" "$LOCAL_HERMES/skills/" 2>/dev/null || true

echo "Checking modified skills..."
diff -rq "$CLONE_DIR/skills" "$LOCAL_HERMES/skills" 2>/dev/null | grep "differ" || echo "No modified skills"

echo "Merging memory..."
python3 -c "
import sqlite3, sys
backup_db = sqlite3.connect('$CLONE_DIR/memory_store.db')
local_db = sqlite3.connect('$LOCAL_HERMES/memory_store.db')
try:
    backup_rows = backup_db.execute('SELECT * FROM memory').fetchall()
    local_rows = local_db.execute('SELECT * FROM memory').fetchall()
    local_set = set()
    for row in local_rows:
        # Use all columns as dedup key
        local_set.add(tuple(row))
    
    added = 0
    for row in backup_rows:
        if tuple(row) not in local_set:
            cols = ','.join(['?' for _ in row])
            local_db.execute(f'INSERT INTO memory VALUES ({cols})', row)
            added += 1
    local_db.commit()
    print(f'Added {added} new memory entries')
except Exception as e:
    print(f'Memory merge error: {e}', file=sys.stderr)
finally:
    backup_db.close()
    local_db.close()
"

echo "Cleanup..."
rm -rf "$CLONE_DIR"

echo "Merge complete at $(date)"
echo "Note: Restart Hermes to reload merged state"

Schedule

Run after each backup push:

• After instance A pushes to backup → instance B runs merge
• Or schedule cron to run merge every 6 hours

# In Hermes cronjob:
# Create job: schedule='every 6h'
# Prompt: 'Run the merge script: bash ~/.hermes/scripts/merge-from-backup.sh'
# no_agent: true