---
name: rag-operations
description: "RAG ops: RBAC permissions, DSPY fixes, deployment."
version: 1.0.0
---

# RAG Operations

SSH target: `rag@rag.opencomputing.cn`, workdir: `/d/rag/ragserver`. DSPY files live under `pkgs/rag/wwwroot/`, UI files alongside them. Local repo: `~/work/rag/rag`, remote: `git@git.opencomputing.cn:yumoqing/rag.git`.

## RBAC Permission Setup (new .dspy → 200 OK)

A 403 on a RAG `.dspy` endpoint means the RBAC middleware rejected the request. The fix requires **BOTH** a `permission` table entry **AND** a `rolepermission` table entry linking it to `roleid='any'`. Either one missing = 403.

### Step 0 — Confirm the DSPY file exists

```bash
ssh rag@rag.opencomputing.cn "find /d/rag/ragserver/pkgs/rag -name '<name>.dspy'"
# Files are under .../wwwroot/... not directly under pkgs/rag/
```

### Step 1 — Diagnose both tables

Use Python heredoc (avoids shell escaping of `%` and quotes):

```bash
ssh rag@rag.opencomputing.cn "cd /d/rag/ragserver && PYTHONPATH='pkgs/sqlor:pkgs/ahserver:pkgs/apppublic' /d/rag/ragserver/py3/bin/python << 'PYEOF'
import sys, asyncio
sys.path.insert(0, 'pkgs/sqlor')
sys.path.insert(0, 'pkgs/ahserver')
sys.path.insert(0, 'pkgs/apppublic')
from appPublic.jsonConfig import getConfig
from sqlor.dbpools import DBPools
async def main():
    conf = getConfig('/d/rag/ragserver')
    db = DBPools(conf['databases'])
    async with db.sqlorContext('rag') as sor:
        # permission table — %% escapes % in LIKE (sqlor uses %-style formatting)
        rows = await sor.sqlExe(
            \"SELECT id, path, permtype FROM permission WHERE path LIKE '%%<name>%%'\",
            ns={})
        print('=== permission table ===')
        for r in rows: print(dict(r))
        if not rows: print('(NONE — needs INSERT)')

        # rolepermission table — column is 'permid' NOT 'permissionid'
        rows2 = await sor.sqlExe(
            \"SELECT * FROM rolepermission WHERE permid IN (SELECT id FROM permission WHERE path LIKE '%%<name>%%')\",
            ns={})
        print()
        print('=== rolepermission table ===')
        for r in rows2: print(dict(r))
        if not rows2: print('(NONE — THIS IS THE PROBLEM even if permission exists!)')
asyncio.run(main())
PYEOF"
```

Replace `<name>` with the DSPY filename stem, e.g. `tag_options`.

### Step 2a — Insert permission (if missing)

```bash
ssh rag@rag.opencomputing.cn "cd /d/rag/ragserver && PYTHONPATH='pkgs/sqlor:pkgs/ahserver:pkgs/apppublic' /d/rag/ragserver/py3/bin/python << 'PYEOF'
import sys, asyncio
sys.path.insert(0, 'pkgs/sqlor')
sys.path.insert(0, 'pkgs/ahserver')
sys.path.insert(0, 'pkgs/apppublic')
from appPublic.jsonConfig import getConfig
from sqlor.dbpools import DBPools
async def main():
    conf = getConfig('/d/rag/ragserver')
    db = DBPools(conf['databases'])
    async with db.sqlorContext('rag') as sor:
        await sor.sqlExe(
            \"INSERT INTO permission (id, path, permtype) VALUES ('<perm_id>', '/rag/knowledge_bases_list/<name>.dspy', NULL)\",
            ns={})
        print('INSERTED permission: <perm_id>')
asyncio.run(main())
PYEOF"
```

### Step 2b — Insert rolepermission (MANDATORY)

Pattern: `roleid='any'` grants access to all authenticated users. This is the most commonly missed step — permission exists but 403 persists because rolepermission is missing.

```bash
ssh rag@rag.opencomputing.cn "cd /d/rag/ragserver && PYTHONPATH='pkgs/sqlor:pkgs/ahserver:pkgs/apppublic' /d/rag/ragserver/py3/bin/python << 'PYEOF'
import sys, asyncio, uuid
sys.path.insert(0, 'pkgs/sqlor')
sys.path.insert(0, 'pkgs/ahserver')
sys.path.insert(0, 'pkgs/apppublic')
from appPublic.jsonConfig import getConfig
from sqlor.dbpools import DBPools
async def main():
    conf = getConfig('/d/rag/ragserver')
    db = DBPools(conf['databases'])
    async with db.sqlorContext('rag') as sor:
        rp_id = 'rp_' + uuid.uuid4().hex[:12]
        await sor.sqlExe(
            \"INSERT INTO rolepermission (id, roleid, permid) VALUES ('\" + rp_id + \"', 'any', '<perm_id>')\",
            ns={})
        print('INSERTED: id=' + rp_id + ', roleid=any, permid=<perm_id>')
        # Verify
        rows = await sor.sqlExe(
            \"SELECT * FROM rolepermission WHERE permid = '<perm_id>'\",
            ns={})
        for r in rows: print('VERIFY:', dict(r))
asyncio.run(main())
PYEOF"
```

### Step 3 — Restart ragserver (RBAC is loaded at startup)

```bash
ssh rag@rag.opencomputing.cn "ps aux | grep 'app/ragserver' | grep -v grep | awk '{print \$2}' | xargs kill; sleep 2; cd /d/rag/ragserver && nohup py3/bin/python app/ragserver.py > /dev/null 2>&1 &"
```

### Step 4 — Verify (expect 200)

```bash
curl -s -o /dev/null -w '%{http_code}' 'https://rag.opencomputing.cn/rag/knowledge_bases_list/<name>.dspy?_webbricks_=1&_lang=zh'
```

## RAG code deployment workflow

Standard flow for RAG module changes:

```bash
# Local: edit files under ~/work/rag/rag/
cd ~/work/rag/rag
git add <files>
git commit -m "description"
git push

# Server: pull. DSPY files auto-reload (hot_reload=True), no restart needed.
ssh rag@rag.opencomputing.cn "cd /d/rag/ragserver/pkgs/rag && git pull"

# Verify with curl
```

SSH background start patterns (when restart is needed):
- `nohup ... &` — hangs SSH; avoid.
- `setsid ... & disown` — works but `disown` may timeout. Acceptable.
- `ssh -f user@host 'cmd'` — reliable background launch.
- `background=true` in Hermes terminal tool — best option within Hermes.

## Schema reference

### permission table
| Column      | Notes |
|-------------|-------|
| id          | Primary key, e.g. `tag_options_perm_001` |
| path        | DSPY path, e.g. `/rag/knowledge_bases_list/tag_options.dspy` |
| permtype    | Usually NULL for DSPY endpoints |

### rolepermission table
| Column      | Notes |
|-------------|-------|
| id          | Primary key, auto-generated |
| roleid      | `'any'` for all authenticated users |
| permid      | Foreign key to permission.id — **column is `permid` not `permissionid`** |

## Pitfalls

- **sqlor LIKE `%` escaping**: sqlor uses Python `%`-style string formatting, so literal `%` in SQL must be doubled: `LIKE '%tag%'` → `LIKE '%%tag%%'`. Single `%` → `TypeError: not enough arguments for format string`.
- **sqlor `IN (${ids}$)` list expansion fails on rag server**: The rag server's sqlor version does NOT auto-expand Python lists for `IN` clauses. `sor.sqlExe("... IN (${ids}$)", {"ids": ["a","b"]})` → `Illegal parameter data types varchar and row`. Workaround: build comma-separated quoted string manually — `id_list = ','.join(["'" + str(x) + "'" for x in doc_ids])` and concatenate into SQL. Pass `ns={}`.
- **DSPY media_tags JOIN crashes**: The standard `WHERE mt.media_id IN (${ids}$)` in media_cards.dspy triggers the sqlor list-expansion bug above. Wrap in try/except so cards still render without tags on failure.
- **KB deletion misses cleanup**: The original `delete_kb.dspy` only deleted DB rows — vector DB (VDB), graph DB, physical files, entities/entity_relations were NOT cleaned up. Physical files were never deleted because `fp.startswith("/idfile/")` never matched actual paths (which are `/44/126/...`). Add VDB delete via `vectordb.opencomputing.net:10443/v1/delete`, graph delete via `localhost:9092/api/graph/delete`, entity/relation cleanup, and fix file path matching to `fp.startswith("/")`.
- **DSPY voice query only matched audio files**: `media_cards.dspy` originally queried by audio extensions (mp3/wav/m4a...) only. Videos (mp4) with extracted voiceprints needed `metadata LIKE '%%voiceprint_status%%done%%'` in the WHERE clause.
- **DSPY debugging technique**: When a .dspy returns generic error like "加载失败", temporarily replace `except Exception: return error_widget` with `except Exception as e: return widget with str(e)` to surface the real error. A single Python syntax error in one line of the DSPY string blocks the entire file execution.
- **rolepermission column name**: It's `permid` (not `permissionid`). Using `permissionid` → `OperationalError: Unknown column 'permissionid'`.
- **403 with permission existing**: Always check `rolepermission` table next — this is the #1 cause of "permission exists but still 403".
- **DSPY file location**: Files live under `pkgs/rag/wwwroot/...` not `pkgs/rag/...`. Use `find` to locate.
- **`${var}$` placeholder syntax**: Works in DSPY files but NOT in direct `sor.sqlExe()` calls from Python scripts. In Python, use string formatting (f-strings or concatenation) for values and pass `ns={}`.
- **Server process name**: The ragserver runs as `app/ragserver.py` (not `sage.py`). Kill with `pkill -f 'app/ragserver'` or the awk pattern above.

### Media search results
- **Media URLs must use `entire_url()`**: `safe_url()` only prepends `/idfile`; use `entire_url(safe_url(path))` → full `https://rag.opencomputing.cn/idfile/...` URL. Without this, VideoPlayer/Image/Audio widgets get relative paths that may not resolve.
- **Video playback**: Use native `<video>` tag (via `Html` widget), NOT bricks `VideoPlayer`. The bricks component has height-collapse issues in search cards. `"<video controls autoplay muted playsinline style=\"width:100%;max-height:400px\" src=\"...\">"` works reliably.
- **Position info display**: Chunks can have position metadata (`bbox`, `start_time`, `end_time`) stored in `document_chunks.metadata` JSON column. Display `📍 (x1,y1)-(x2,y2)` for face bounding boxes, `⏱ 3.2s → 8.5s` for video/audio timestamps.
- **Voiceprint API response**: `POST /voiceprint/extract/submit` returns `{"status":"SUCCEEDED","embedding":[...],"embedding_dim":192}` — NO `speakers` field. Check `embedding_dim > 0` for success, not `speakers`.
- **Tag display**: Tags stored in TWO places — `media_tags` table (from processing) AND `documents.metadata.tags` JSON array (from add_tag UI). Display logic must read both sources and deduplicate.
- **DSPY boolean**: Use Python `True`/`False` not JS `true`/`false`. `"autoplay": true` → `NameError`. JSON serialization converts Python bools automatically.