---
name: rbac-permission-initialization-pattern
description: Pattern for initializing RBAC permissions in business modules that use wildcard expansion, role ID matching, and path registration
author: Hermes Agent
tags: [rbac, permissions, init, multi-tenant, sage]
---

# RBAC Permission Initialization Pattern

## Overview
When deploying a business module with RBAC authentication, the permission initialization must handle several complex scenarios: path normalization, role wildcard expansion, CRUD file structure, and URL rewriting edge cases (WSS, index auto-match).

**User preference**: Each business module owns its own `scripts/load_path.py` that registers its paths directly via Sage DB operations (mirroring Sage's `load_path.py` internals). This keeps permissions self-contained per module and runnable from any Sage environment. See `references/per-module-load-path.md` for the Python template pattern. The Sage-level `load_path.py` is the canonical declarative source of truth, but for per-module workflows, the module's own script is preferred.

## Role-Based Permission Analysis Methodology

Before writing permission scripts, analyze each role's responsibilities and classify paths into tiers:

**Step 1: Document role职责**
```
owner.superuser    — 系统级: 机构类型/角色/权限管理, 添加业主管理员
*.admin            — 机构级: 添加本机构人员, 分配人员角色
reseller.operator  — 运营: 产品管理/供应商合同/定价/统一折扣/营销
reseller.sale      — 销售: 客户管理/客户特殊折扣
reseller.accountant — 财务: 线下充值/对账结算
reseller.maintainer — 运维维护
customer.customer  — 终端客户用户
logined            — 所有已登录用户
```

**Step 2: Analyze module business nature**
- Ask: Is this a domain-specific business module (CRM, accounting) or a general tool service (AI agent, reasoning)?
- General tool services → broader access (all logined users can use)
- Domain-specific modules → restricted to relevant roles

**Step 3: Classify into permission tiers**
| Tier | Role set | Path types |
|------|----------|-----------|
| **Public** | `any` | 登录/注册/认证页面、静态资源(img/css) |
| **Logined** | all 登录角色 | 用户自助服务(个人信息、API Key)、数据查看(列表+get)、用户自己的CRUD(通过user_id隔离) |
| **Admin** | superuser + *.admin | 系统/机构配置管理、用户管理、机构管理 |
| **Superuser** | `owner.superuser` only | 全局元数据(角色/权限/机构类型)、高危操作(技能部署) |

**Step 4: Register CRUD paths comprehensively**
- JSON CRUD `alias` → directory `{alias}/` with `index.ui`, `get_*.dspy`, `add_*.dspy`, `update_*.dspy`, `delete_*.dspy`
- Custom `api/` directory may also contain CRUD endpoints — register both
- Every CRUD directory needs TWO paths (see Pitfall 2)

## Key Principles

### 1. Wildcard Expansion (Filesystem Scanning)
**Problem**: `rbac.check_roles_path()` does exact matching. Permission definitions must be registered as concrete URLs in the DB.

**CRITICAL**: Scan ALL file types — `.ui`, `.dspy`, `.js`, `.css`. RBAC protects all static resources.

**IMPORTANT: ahserver auto-serves `.css` and `.js` files** from module `wwwroot/` directories — they are injected into HTML responses without explicit `<link>`/`<script>` tags in `.ui` files. **Do NOT manually reference these files in `.ui` widgets.** However, they are still subject to RBAC checks. Every `.css`/`.js` file must be registered in the permission system. Theme files needed before auth should use `any` role.

**Symlink handling**: Use `followlinks=True` in `os.walk`, but guard against self-referencing symlinks (e.g., `main -> wwwroot`):
```python
real_wwwroot = os.path.realpath(wwwroot_path)
visited_real = set()
for root, dirs, files in os.walk(wwwroot_path, followlinks=True):
    real_root = os.path.realpath(root)
    if real_root in visited_real:
        dirs.clear()
        continue
    visited_real.add(real_root)
    dirs[:] = [d for d in dirs if os.path.realpath(os.path.join(root, d)) != real_wwwroot]
```

### 2. Directory URL Dual Path Registration (including root `/`)
**Problem**: User visits a directory URL and gets 401, even though the `index.ui` inside has permission.

**Root Cause**: ahserver's `website.indexes` config auto-matches `index.ui` for directory URLs. RBAC checks the **raw request path**, NOT the resolved file path. When the user visits `/harnessed_agent/hermes_memory`, RBAC sees `/harnessed_agent/hermes_memory` (NO `/index.ui` suffix). Same applies to root: visiting `http://host/` checks path `/`, NOT `/index.ui`.

**Fix**: Register BOTH the directory path AND the index file path:
```bash
# CRUD subdirectory — register both:
python set_role_perm.py "logined" "/harnessed_agent/hermes_memory"
python set_role_perm.py "logined" "/harnessed_agent/hermes_memory/index.ui"

# ROOT path — same pattern, register both:
python set_role_perm.py "any" "/"
python set_role_perm.py "any" "/index.ui"
```

**Pitfall for auto-discovery scripts**: When scanning wwwroot files to auto-register permissions (e.g., `init_any_permissions.py`), always explicitly append `/` to the path list — it's not a real file so `os.listdir()` won't find it:
```python
root_paths = ["/" + f for f in os.listdir(wwwroot_dir) if ...]
root_paths.append("/")  # CRITICAL: root path is not a file
```

### 3. Path Registration (NO /main prefix)
**CRITICAL**: Do NOT add `/main` prefix to permission paths. The `config.json` `paths` mapping handles URL routing at the HTTP level. RBAC permission checks use paths relative to wwwroot.

```python
def normalize_path(path):
    if path.startswith('/main'):
        cleaned = path[5:]
        return cleaned if cleaned else '/'
    return path
```

### 4. WSS WebSocket Path Handling
**CRITICAL**: Server logs confirm RBAC checks the FULL path INCLUDING `/wss/` prefix. Both UI and RBAC use the same path format.

**Verified from Sage server logs**: `path='/wss/harnessed_reasoning/reasoning_console.wss' permission check failed`

**Rule**:
- **UI reference**: `{{entire_url('/wss/module/file.wss')}}` — full path with `/wss/`
- **RBAC permission**: `/wss/module/file.wss` — SAME path, WITH `/wss/` prefix
- **set_role_perm.py**: `python set_role_perm.py "logined" "/wss/module/file.wss"`

### 5. ahserver .wss Processor Mapping
**Symptom**: WebSocket to `.wss` endpoint returns 401 or is handled as a static file. Log: "handle as a normal file".
**Root cause**: `conf/config.json` `processors` list only has `[".ws","ws"]`, not `[".wss","ws"]`.
**Fix**: Add `[".wss","ws"]` to the `processors` array in `conf/config.json`:
```json
"processors":[
    [".ws","ws"],
    [".wss","ws"],
    ...
]
```

### 6. RBAC 'any' and 'logined' Roles Require role Table Entries
**Symptom**: Users get 401 despite `rolepermission` having correct entries.
**Root cause**: `userperm.py` joins `rolepermission` + `permission` + `role` tables. Missing `role` rows mean no results for those keys.

**Critical roles that MUST exist in `role` table:**
```sql
-- Anonymous users (public resources)
INSERT INTO role (id, orgtypeid, name) VALUES ('any', '*', 'any');

-- Authenticated users (module access after login)
INSERT INTO role (id, orgtypeid, name) VALUES ('logined', '*', 'logined');
```

**Why `logined` is critical**: `userperm.py` automatically assigns `['any', 'logined']` to all authenticated users. If the `logined` role doesn't exist in the `role` table, logged-in users have NO permissions despite having valid sessions.

**Detection**: If login succeeds but all module paths return 401:
```sql
-- Check if logined role exists
SELECT * FROM role WHERE name='logined';
-- If empty, create it and register paths
```

**Fix**: Ensure both roles exist and paths are registered in `load_path.py`:
```python
PATHS_ANY = ["/", "/index.ui", "/menu.ui", "/bricks/**", "/rbac/user/login.ui"]
PATHS_LOGINED = ["/module/**", "/module/api/%", "/module/index.ui", ...]
```

### 7. role Table Missing name Column
**Symptom**: `Unknown column 'c.name' in 'SELECT'` when RBAC loads role permissions.
**Root cause**: DDL from xlsx may not include `name` column that `userperm.py` expects.
**Fix**: `ALTER TABLE role ADD COLUMN name VARCHAR(100);` then populate from `role` field.

### 8. Role Wildcard Matching (orgtypeid='*' 通配所有机构)
`get_userroles()` expands roles into THREE keys:
```python
roles.append(f'{r.orgtypeid}.{r.name}')   # e.g., 'customer.sales_rep'
roles.append(f'{r.orgtypeid}.*')           # e.g., 'customer.*'
roles.append(f'*.{r.name}')                # e.g., '*.sales_rep'
```
Create roles with `orgtypeid='*'` — RBAC matches across all organizations via `*.{name}`.

### 9. Core Framework UI Files Must Be `any` Role
**Symptom**: Visiting the root path `/` or `index.ui` returns HTTP 401, even though login works and module permissions are set. The app appears completely inaccessible.

**Root Cause**: Sage's root page loads several core framework files (top.ui, center.ui, bottom.ui, bricks CSS/JS, public assets). If these are not registered for the `any` role, anonymous users cannot load the shell UI — even though they could theoretically log in, the framework files they need to see the login page are blocked.

**Required `any` permissions for every Sage deployment**:
```bash
# Core UI shell (the app frame)
python set_role_perm.py "any" "/"
python set_role_perm.py "any" "/index.ui"
python set_role_perm.py "any" "/top.ui"
python set_role_perm.py "any" "/center.ui"
python set_role_perm.py "any" "/bottom.ui"

# Bricks framework (ALL files recursively — CSS, JS, imgs, 3parties, examples, docs)
find wwwroot/bricks -type f | while read f; do
    rel="${f#wwwroot}"
    python set_role_perm.py "any" "$rel"
done

# Public assets (ALL files recursively)
find wwwroot/public -type f | while read f; do
    rel="${f#wwwroot}"
    python set_role_perm.py "any" "$rel"
done

# Common navigation/menu files
python set_role_perm.py "any" "/user_menu.ui"
python set_role_perm.py "any" "/menu.ui"
python set_role_perm.py "any" "/accordion.ui"
python set_role_perm.py "any" "/app_panel.ui"
```

**Verification**: After setting permissions, restart the app and test:
```bash
curl -s -o /dev/null -w '%{http_code}' http://localhost:9180/              # 200
curl -s -o /dev/null -w '%{http_code}' http://localhost:9180/bricks/css/bricks.css  # 200
curl -s -o /dev/null -w '%{http_code}' http://localhost:9180/bricks/imgs/app.svg     # 200
curl -s -o /dev/null -w '%{http_code}' http://localhost:9180/public/index.ui         # 200
```

**Important**: Module-level `wwwroot` directories that live OUTSIDE the sage wwwroot (e.g., `harnessed_agent`, `harnessed_reasoning` in `~/repos/`) must be symlinked into `sage/wwwroot/` so ahserver can serve them:
```bash
cd ~/repos/sage/wwwroot
ln -s ../../harnessed_agent/wwwroot harnessed_agent
ln -s ../../harnessed_reasoning/wwwroot harnessed_reasoning
```

### 12. RBAC Tools Placement and .dspy Function Pattern

**User correction**: RBAC utility tools (path-permission queries, unauth file scans, etc.) MUST live in the `rbac` module (`~/repos/rbac/wwwroot/`), NOT in `sage/scripts/` or any other module's scripts directory.

**CRITICAL RULE: `.dspy` files MUST NOT contain `import` statements or f-strings with braces in dict literals.**

`import` is prohibited because `.dspy` runs in a restricted `exec()` namespace. All logic with imports must live in the module's Python files.

**f-string braces `{var}` inside dict literals cause `SyntaxError: '{' was never closed`.** The `exec()` environment misparses `{e}` inside `f"..."` as a dict brace. Use string concatenation instead:
```python
# WRONG — SyntaxError in .dspy
return {"widgettype": "Message", "options": {"message": f"处理失败: {e}"}}

# CORRECT — use concatenation
return {"widgettype": "Message", "options": {"message": "处理失败: " + str(e)}}
```
The `debug(f'...')` / `exception(f'...')` calls are fine since they don't nest inside dict braces.

This includes `import os`, `import json`, `from collections import defaultdict`, `from appPublic.dictObject import DictObject`, etc. The `.dspy` file is executed in an inline eval context with a limited namespace — imports are prohibited. All logic with imports must live in the module's Python files.

**Correct pattern for RBAC utility tools (and any non-trivial .dspy):**

1. **`rbac/<module>/tools.py`** — Python module with imports and all business logic. Functions return `(title, message, is_error)` tuples.
2. **`rbac/<module>/init.py`** — Register functions on ServerEnv in `load_rbac()`:
   ```python
   from .tools import query_path_roles, scan_unauth_files
   # In load_rbac():
   env.query_path_roles = query_path_roles
   env.scan_unauth_files = scan_unauth_files
   ```
3. **`wwwroot/<tool>.ui`** — ModalForm UI for user input
4. **`wwwroot/<tool>.dspy`** — Zero imports. Call module functions via `request._run_ns`:
   ```python
   async with get_sor_context(request._run_ns, 'rbac') as sor:
       title, message, is_error = await request._run_ns.query_path_roles(sor, path)
   if is_error:
       return UiError(title=title, message=message)
   return UiMessage(title=title, message=message)
   ```
5. **`wwwroot/admin_menu.ui`** — Add menu entry (accessed via /appbase/menu.ui for superuser)
6. **Register permissions** — Both .ui and .dspy paths need `owner.superuser` role

**Function return convention**: Module functions should return `(title, message, is_error)` tuples. The `.dspy` file unpacks these and decides between `UiError` and `UiMessage`. This keeps HTML rendering logic in the Python module where imports are allowed.

**Standalone .py → .dspy conversion rules:**
- Input: replace `sys.argv` → `params_kw.get('name', '')`
- DB context: replace `DBPools(config.databases)` + `sqlorContext('sage')` → `get_sor_context(request._run_ns, 'rbac')`
- Output: replace `print()` → `return UiMessage(title='...', message='...')` or `UiError`
- `sor.sqlExe()` always requires `ns` dict, even with zero placeholders: `sqlExe("SELECT ...", {})` — passing without `{}` raises `TypeError: missing 1 required positional argument: 'ns'`

**DSPY→Python delegation pattern**: When a `.dspy` needs complex logic, put it in a Python method on the provider/class, accessible via `env.PROVIDERS`:
1. Add the method to the provider's `.py` file — does the work, returns `(title, message)`
2. Ensure the provider is registered on `env.PROVIDERS` (usually already done in `load_xxx()`)
3. `.dspy` just calls: `provider = env.PROVIDERS.get('xxx'); title, msg = await provider.method(params, env)`

This keeps the `.dspy` thin (no imports, no complex logic, no f-strings in dicts) while Python code has full import access.

**DSPY→Python delegation pattern**: When a `.dspy` needs complex logic (email scanning, file I/O, API calls), put the logic in a Python method and register it on ServerEnv. The `.dspy` just calls the method:

### 9a. Cron DSPY Endpoints: `any` Role + IP Whitelist Pattern

**Pattern**: External crontab triggers `.dspy` endpoints via `curl localhost`. Since cron has no auth token, these endpoints MUST use `any` role in RBAC. Security is enforced by IP whitelist in the Python function, NOT by RBAC.

**Architecture**:
```
crontab → curl localhost → .dspy (any role) → init.py function (IP check)
```

**load_path.py** — register cron paths as `any`:
```python
PATHS_ANY = [
    f"/{MOD}/cron/etl_sync.dspy",
    f"/{MOD}/cron/etl_aggregate.dspy",
    f"/{MOD}/cron/etl_provider_cost.dspy",
]
```

**init.py** — registered function with IP whitelist:
```python
async def cron_etl_sync(request):
    ip = request['client_ip']
    if ip not in ['127.0.0.1']:
        return {'error': 'IP ' + ip + ' not allowed'}
    env = request._run_ns
    async with get_sor_context(env, MODULE_NAME) as sor:
        count = await sync_call_fact(sor)
        return {'status': 'ok', 'synced': count}

# In load_module():
env.cron_etl_sync = cron_etl_sync
```

**`.dspy` thin wrapper** (NO imports, NO f-strings in dicts):
```python
# ETL: description of what this does
result = await request._run_ns.cron_etl_sync(request)
return str(result)
```

**build.sh crontab entry** (guard against duplicates):
```bash
dm_cron="*/5 * * * * curl -s http://localhost:9180/${MOD}/cron/etl_sync.dspy"
if ! echo "$cc"|grep -Fq "etl_sync"; then
    (echo "$cc";echo "$dm_cron") | crontab -
fi
```

**Pitfall**: Never put `import` or `from` in cron `.dspy` files — the exec() namespace prohibits it. All imports and logic must live in init.py registered functions.

**Pitfall**: Cron endpoints use `any` role (not `logined`) because curl from crontab has no session cookie. The IP whitelist (`request['client_ip']`) in the Python function is the security boundary.

### 9b. Login DSPY Endpoints Must Be `any` Role (CRITICAL)

**Symptom**: Login page loads (200) but submitting login returns 401 Unauthorized. User sees "401: Unauthorized" text instead of being redirected to the app.

**Root Cause**: The login UI pages (login.ui, userpassword_login.ui) are registered as `any`, but the dspy endpoints that **process** the login POST requests are NOT. RBAC blocks the POST before authentication can happen.

**Required `any` permissions for ALL login-related dspy endpoints:**
```python
PATHS_ANY = [
    # Login UI pages (already known)
    "/rbac/user/login.ui",
    "/rbac/user/register.ui",
    "/rbac/userpassword_login.ui",
    # Login DSPY endpoints (MUST ALSO be any!)
    "/rbac/userpassword_login.dspy",     # password login POST handler
    "/rbac/phone_login.dspy",            # phone login POST handler
    "/rbac/user/code_login.dspy",        # SMS code login POST handler
    "/rbac/gen_sms_code.dspy",           # send SMS verification code
    "/rbac/user/register.dspy",          # user registration POST handler
    "/rbac/user/sms_register.dspy",      # SMS registration POST handler
    "/rbac/user/up_login.dspy",          # session-based login
    "/rbac/user/logout.dspy",            # logout
]
```

**Detection**: `curl -s -o /dev/null -w "%{http_code}" http://host/rbac/userpassword_login.dspy` returns 401 → missing `any` permission.

**Fix**: Register all login dspy paths as `any` in load_path.py or RBAC init script.

### 9c. Independent App `set_role_perm.py` Wrapper

**Context**: Independent apps (pipeline-app, etc.) don't have Sage's root-level `set_role_perm.py`. Module `load_path.py` scripts call `py3/bin/python set_role_perm.py <role> <path>` which expects the script at app root.

**Solution**: Create `set_role_perm.py` at app root as a thin wrapper:
```python
#!/usr/bin/env python3
"""Wrapper for RBAC permission registration. Called by modules' load_path.py."""
import sys, os, asyncio
sys.path.insert(0, os.path.dirname(os.path.abspath(__file__)))
from rbac.set_role_perms import set_role_perm as _set_role_perm
from appPublic.jsonConfig import getConfig
from sqlor.dbpools import DBPools

async def main(role, path):
    parts = path.strip('/').split('/')
    module = parts[0] if parts else 'app'
    config = getConfig('.', {'workdir': '.'})
    db = DBPools(config.databases)
    await _set_role_perm('APP_DBNAME', module, '*', role, path)

def run(coro):
    loop = asyncio.new_event_loop()
    asyncio.set_event_loop(loop)
    try: loop.run_until_complete(coro)
    finally: loop.close()

if __name__ == '__main__':
    if len(sys.argv) != 3:
        print(f"Usage: {sys.argv[0]} <role> <path>"); sys.exit(1)
    run(main(sys.argv[1], sys.argv[2]))
```

**Load_path.py universal pattern** (works for any app, not just Sage):
```python
def register_paths():
    for path in PATHS_ANY:
        subprocess.run(["py3/bin/python", "set_role_perm.py", "any", path])
    for path in PATHS_LOGINED:
        subprocess.run(["py3/bin/python", "set_role_perm.py", "logined", path])
```

**Pitfall**: Do NOT hardcode Sage paths or `SAGE_ROOT` detection in load_path.py. The script runs from the app's root directory using the app's own venv.

### 9d. Async Startup Tasks MUST Use add_startup, NOT asyncio.ensure_future

**Symptom**: `Task was destroyed but it is pending! coroutine 'X.run' was never awaited` on startup.

**Root Cause**: Module `init()` functions run SYNCHRONOUSLY before the event loop starts (called from `configuredServer.py`). `asyncio.ensure_future()` or `asyncio.create_task()` in init phase creates tasks that are never scheduled.

**Fix**: Use `add_startup(coro)` from `ahserver.configuredServer` — it registers a callback that runs once when the event loop starts:
```python
from ahserver.configuredServer import add_startup

def load_module():
    for name in providers:
        PROVIDERS[name] = get_provider(name, conf)
        if name == 'transfer':
            add_startup(PROVIDERS[name].run)  # ✓ runs after event loop starts
            # NOT: asyncio.ensure_future(PROVIDERS[name].run())  # ✗ init phase, no event loop
```

### 10. Permission Cache Refresh
RBAC caches permissions in memory. Must restart the app after DB changes.

**IMPORTANT: ALL requests pass through RBAC middleware** — even static files (`.css`, `.js`, `.png`, `.svg`). There is no static file bypass. If the RBAC database is unreachable, **every single request returns 500**, including all static resources. See `references/ahserver-auth-middleware.md` for the full middleware architecture, common failure modes, and cache details.

### 10b. Script Output Silencing for Large Batches
**Symptom**: `set_role_perm.py` scripts with hundreds of paths time out or flood the terminal.
**Root cause**: Each permission registration prints output (e.g., `logined, xxxxx perm add` or `already exists`). With 1500+ permission entries, the output buffer overflows.
**Fix**: Redirect output in the `set_perm` function:
```bash
set_perm() {
    local role="$1"
    local path="$2"
    python set_role_perm.py "${role}" "${path}" > /dev/null 2>&1
    COUNT=$((COUNT + 1))
}
```

### 10c. set_core_perms.sh Pattern for Static Assets
Sage uses `set_core_perms.sh` to batch-register `any` role permissions for static assets (bricks, public, core UI). The script uses a helper function and `find`:
```bash
COUNT=0
set_perm() {
    python set_role_perm.py "$1" "$2" > /dev/null 2>&1
    COUNT=$((COUNT + 1))
}
for p in "/index.ui" "/top.ui" "/center.ui" "/bottom.ui"; do
    set_perm "any" "$p"
done
while IFS= read -r file; do
    rel="${file#/home/hermesai/repos/sage/wwwroot}"
    set_perm "any" "$rel"
done < <(find wwwroot/bricks -type f 2>/dev/null)
```

### 10d. Wildcard % and ** as First-Class Permission Standards

**User correction**: Manually listing every `.dspy` file in `load_path.py` leads to 403 errors when new files are added but the script isn't updated. Use `%` or `**` wildcards to auto-cover directories.

**Two wildcard suffixes are supported** by `userperm.check_roles_path()`:
- `%` suffix — SQL LIKE-style wildcard (e.g., `/module/api/%` matches `/module/api/anything`)
- `**` suffix — glob-style wildcard (e.g., `/module/**` matches `/module/anything`)

Both produce identical behavior: prefix match against the request path. Use whichever style you prefer, but be consistent within a project.

**Standard patterns in load_path.py:**
```python
# api/ directory — covers all .dspy endpoints
f"/{MOD}/api/%",

# CRUD subdirectories — covers add/get/update/delete .dspy + index.ui
f"/{MOD}/llm/%",
f"/{MOD}/llmcatelog/%",

# Loose top-level .dspy files (non-api, non-CRUD)
f"/{MOD}/%.dspy",

# Static assets
f"/{MOD}/imgs/%",
```

**When to list files explicitly vs. wildcards:**
- ✅ Wildcard: directories with many files that grow over time (`api/`, CRUD dirs, `imgs/`)
- ✅ Wildcard: loose `.dspy` at top level (`/{MOD}/%.dspy`)
- ✅ Explicit: specific `.ui` pages that need role-based access (e.g., admin-only pages)
- ✅ Explicit: callback endpoints that need `any` role (e.g., `api/rl_callback.dspy`)

**Role-based separation example (reallife_asset pattern):**
```python
# logined — customer-visible pages + all APIs (scripts do get_user() checks internally)
PATHS_LOGINED = [
    f"/{MOD}",
    f"/{MOD}/index.ui",
    f"/{MOD}/create_validate.ui",    # customer page
    f"/{MOD}/upload_asset.ui",       # customer page
    f"/{MOD}/api/%",                 # all API endpoints
]

# operator — admin-only pages (separate list for specific roles)
PATHS_OPERATOR = [
    f"/{MOD}/group_manage.ui",
    f"/{MOD}/vendor_config_manage.ui",
    f"/{MOD}/asset_manage.ui",
]
```

### 10e. Wildcard SQL LIKE for Bulk Permission Checks
To check or register permissions using SQL LIKE patterns, use `set_role_perm.py` with `%` wildcard:
```bash
# Register all /bricks/* paths (matches existing records via LIKE)
./py3/bin/python set_role_perm.py any /bricks/%
```
Note: `set_role_perm.py` internally uses `path LIKE ${path}$` SQL when `%` is present in the path argument. Only matches paths already in the permission table — use `find` + loop for newly-added files.

### 11b. sage/wwwroot/menu.ui is gitignored

Sage's `.gitignore` includes `wwwroot/` because most files are symlinks from module repos. The main `wwwroot/menu.ui` is a regular file but is still ignored. Changes to it must be committed with `git add -f wwwroot/menu.ui` or kept local. RBAC module's own `admin_menu.ui` lives in `~/repos/rbac/wwwroot/admin_menu.ui` and IS tracked by git.

### 11c. Standalone Superuser Init Scripts

For modules that need a quick way to grant `owner.superuser` all permissions (e.g., after adding new endpoints or during initial setup), create a standalone `init_superuser.py` at the module root.

**CRITICAL: `py` path must come from `sage_root`, NOT `app_root`.** Multi-module repos (like CMS) often lack their own py3 virtualenv — using `app_root`'s python will fail with `FileNotFoundError`. Always construct: `py = os.path.join(sage_root, "py3", "bin", "python")`.

**Pattern:**
```python
# init_superuser.py — grant owner.superuser all module permissions
import os, sys, subprocess

sage_root = None
for c in [os.path.expanduser("~/repos/sage"), os.path.expanduser("~/sage")]:
    if os.path.isdir(os.path.join(c, "py3", "bin")):
        sage_root = c
        break
if not sage_root:
    print("ERROR: Sage not found"); sys.exit(1)

py = os.path.join(sage_root, "py3", "bin", "python")  # MUST be sage_root, not app_root
sp = os.path.join(sage_root, "set_role_perm.py")

def run(role, paths):
    for p in paths:
        subprocess.run([py, sp, role, p], cwd=sage_root, capture_output=True)

superuser_paths = [
    "/module/index.ui",
    "/module/api/%",
    # ... all paths
]

run("owner.superuser", superuser_paths)
```

**When to use:**
- After manually adding new endpoints to production and need to grant superuser access
- Initial module setup when superuser needs full access before other roles are configured
- Quick permission refresh for superuser after schema changes

**Difference from load_path.py:**
- `load_path.py` defines all roles and their permissions (any, logined, operator, etc.)
- `init_superuser.py` is a convenience script specifically for granting superuser all paths
- Both use `set_role_perm.py` but serve different purposes

**Pitfall: Naming conflict with scripts/init_superuser.py**
Some repos have TWO scripts with similar names:
- `scripts/init_superuser.py` — creates a superuser account in the database (INSERT INTO users)
- Root `init_superuser.py` — grants owner.superuser role all RBAC paths

This causes confusion and build.sh may reference the wrong one. **Fix:** Rename the root-level RBAC script to `init_superuser_permissions.py` to disambiguate. Always use distinct names for scripts that do different things (user creation vs. permission assignment).

### 11d. Auto-Discovery any Permission Script (init_any_permissions.py)

For multi-module repos where `wwwroot/` directories contain many files that change over time, create an auto-discovery script that scans filesystem and sets `any` permissions dynamically:

**Key design points:**
- Scan each module's `wwwroot/` with `os.walk()`, mapping to correct URL prefix
- **Always explicitly append `"/"` to root paths** — it's not a file so filesystem scanning won't find it, but RBAC checks raw request path `/` when users visit the root URL
- Skip symlinks pointing outside the repo (prevents registering other modules' files)
- Skip `__pycache__`, `.git`, `.pyc`, `.bak` etc.
- Handle `bricks/` symlink separately (created during build.sh, may not exist)
- Check symlink targets with `os.path.realpath()` to detect external links

**Multi-module URL prefix mapping:**
```
entcms/wwwroot/*        → /<file>           (no prefix, root-level)
dingdingflow/wwwroot/*  → /dingdingflow/<file>  (with module prefix)
bricks/*                → /bricks/<file>    (framework prefix)
```

**Symlink safety pattern:**
```python
if os.path.islink(full_path):
    link_target = os.path.realpath(full_path)
    if not link_target.startswith(app_root):
        continue  # skip external symlinks
```

### 11e. CRUD UI Files Generated by build.sh

CRUD management pages (e.g., `cms_content_list/index.ui`) do NOT exist in the repo — they are generated by `xls2ui` during `build.sh`. Before build:
- `json/*.json` has CRUD definitions
- `wwwroot/` has hand-written `.ui` files (index, admin, menu, etc.)

After build:
- `wwwroot/<alias>/index.ui` + CRUD `.dspy` files are generated by xls2ui
- These generated files also need RBAC permissions (wildcard `%` handles this)

**Pitfall**: `load_path.py` may reference files like `products.ui` that don't exist yet (referenced but not created). Verify file existence before registering permissions.

### 12. Bulk Module Scanning for load_path.py

When auditing all repos to find which modules need `scripts/load_path.py`:

```bash
# Step 1: Find modules WITH web content but WITHOUT load_path.py
for d in ~/repos/*/; do
  mod=$(basename "$d")
  mdir="$d/wwwroot"
  [ ! -d "$mdir" ] && continue
  has_ui=$(find "$mdir" -name "*.ui" -type f | head -1)
  has_lp="NO"; [ -f "$d/scripts/load_path.py" ] && has_lp="YES"
  [ -n "$has_ui" ] && [ "$has_lp" = "NO" ] && echo "NEEDS: $mod"
done

# Step 2: For each module, collect all web paths
# Scan wwwroot/ for .ui/.dspy files, json/ for CRUD alias dirs
find ~/repos/MODULE/wwwroot -type f \( -name "*.ui" -o -name "*.dspy" \) | sort
ls ~/repos/MODULE/json/*.json 2>/dev/null  # CRUD alias directory names
```

**Path classification rules:**
- `any` role: `menu.ui`, `usermenu.ui`, `admin_menu.ui`, login/register/logout pages, password reset
- `logined` role: everything else (module entry `/mod`, all `.ui`/`.dspy` files, `api/` endpoints, CRUD alias dirs)
- Infrastructure modules without `wwwroot/` do NOT need load_path.py: ahserver, apppublic, bricks-for-python, sqlor, checklang, bench_static
- Note: appbase, dapi, uapi, msp DO have wwwroot/ and need load_path.py

**Template for generating load_path.py**: See `references/per-module-load-path.md`

### 13. set_role_perm.py Role Name Must Be `orgtypeid.name` Format (CRITICAL)

**Problem:** `load_path.py` passes `"developer"` to `set_role_perm.py`, which crashes at line 56: `orgtypeid, name = role.split('.')` — `ValueError: not enough values to unpack`.

**Root Cause:** `set_role_perm.py` only accepts three special role names (`anonymous`, `any`, `logined`) or the `orgtypeid.name` format. Any other format crashes silently (subprocess captures stderr).

**Wrong:**
```python
total += register_role_paths("developer", PATHS_DEVELOPER)
total += register_role_paths("admin", PATHS_ADMIN)
```

**Correct:**
```python
total += register_role_paths("owner.developer", PATHS_DEVELOPER)
total += register_role_paths("owner.superuser", PATHS_SUPERUSER)
total += register_role_paths("customer.admin", PATHS_CUSTOMER_ADMIN)
```

**Detection:** If `load_path.py` runs without error output but permissions aren't registered, check subprocess stderr — the `split('.')` error is captured silently by `subprocess.run(capture_output=True)`.

### 14. Independent App RBAC Database Mismatch (CRITICAL)

**Symptom**: Permissions exist in the database (`rolepermission` + `permission` tables have correct entries), app is restarted, but all paths return 401 Unauthorized. Log shows: `userid=None, path='/' permission check failed`.

**Root Cause**: `set_role_perm.py` hardcodes `db.sqlorContext('sage')` — it always writes to the `sage` database. But independent apps (like CMS) have their own `get_module_dbname()` that returns a different database (e.g., `ocai_cms`). The RBAC check reads from the app's database (`ocai_cms`), but permissions were written to `sage`.

**Write → sage, Read → ocai_cms = MISMATCH**

**Fix**: `set_role_perm.py` supports the `SAGE_RBAC_DB` environment variable:
```bash
# For independent apps, set the target database:
SAGE_RBAC_DB=ocai_cms py3/bin/python ~/sage/set_role_perm.py any /
SAGE_RBAC_DB=ocai_cms py3/bin/python ~/sage/set_role_perm.py owner.superuser /admin.ui
```

**In init scripts**, pass the env var via subprocess:
```python
env = os.environ.copy()
env['SAGE_RBAC_DB'] = 'ocai_cms'  # app's database name
subprocess.run([py, sp, role, p], cwd=sage_root, capture_output=True, env=env)
```

**Default behavior unchanged**: When `SAGE_RBAC_DB` is not set, `set_role_perm.py` defaults to `sage` — no impact on existing Sage deployments.

**Detection**: If `permission` table has the path but RBAC still returns 401, check which database has the data:
```sql
-- In the APP's database (e.g., ocai_cms):
SELECT a.*, b.path FROM rolepermission a, permission b WHERE a.permid = b.id AND b.path='/';
-- If empty here but set_role_perm.py said "already exists" → it was written to sage DB
```

### 15. Menu Submenu Must Point to Correct Module
When adding submenu entries in the main `menu.ui`, ensure each submenu points to its OWN module's `menu.ui`. Two different top-level menu items pointing to the same submenu file will show identical content, hiding the second module's entries entirely.

```json
// WRONG — "推理" shows same content as "代理":
{"name": "agent",    "submenu": "{{entire_url('/harnessed_agent/menu.ui')}}"},
{"name": "reasoning", "submenu": "{{entire_url('/harnessed_agent/menu.ui')}}"},

// CORRECT — each points to its own module:
{"name": "agent",    "submenu": "{{entire_url('/harnessed_agent/menu.ui')}}"},
{"name": "reasoning", "submenu": "{{entire_url('/harnessed_reasoning/menu.ui')}}"},
```

## Assigning User Roles: Critical Pitfalls

### `sor.R('role', ...)` MUST use `orgtypeid` + `name`, NOT a fake `role` field

The `role` table model has only three fields: `id`, `orgtypeid`, `name`. There is NO `role` column.

**WRONG** — `sor.R` ignores unknown fields, returns ALL roles, `role_recs[0]` is random:
```python
# Returns 27 records — first one might be owner.* instead of reseller.admin!
role_recs = await sor.R('role', {'role': 'reseller.admin'})
```

**RIGHT** — split into model fields:
```python
role_recs = await sor.R('role', {'orgtypeid': 'reseller', 'name': 'admin'})
# Returns exactly 1 record: reseller.admin
```

**For special pseudo-roles** (`any`, `logined`, `anonymous`), use `id`:
```python
role_recs = await sor.R('role', {'id': 'logined'})
```

### `logined` and `any` are auto-added — never manually assign

`get_userroles()` in `userperm.py` always prepends `['any', 'logined']` to every authenticated user's role list. **Do not insert `userrole` entries for `logined` or `any`** — it's redundant at best, and at worst can cause confusion when debugging role issues.

Only assign specific `{orgtypeid}.{name}` roles like `reseller.admin`, `reseller.operator`, etc.

### Role dropdown filtering by user org_type

When creating a role dropdown (e.g. for userrole CRUD), filter by the current user's `org_type`. A `reseller` admin should only see `reseller.*` roles, not `owner.*` or `customer.*`.

```python
userorgid = await get_userorgid()
org_recs = await sor.R('organization', {'id': userorgid})
org_type = org_recs[0].org_type if org_recs else ''

# Filter to user's org_type only (org=0 sees all)
recs = await sor.sqlExe(
    "SELECT id, CONCAT(orgtypeid, '.', name) as name FROM role "
    "WHERE id NOT IN ('anonymous', 'any', 'logined') "
    "AND orgtypeid = ${ot}$ AND name != '*' AND name IS NOT NULL",
    {'ot': org_type})
```

Use `get_search_roleid.dspy` pattern with `dataurl` override in CRUD JSON `alters`.

## Role Permission Table Operations

The `rolepermission` table requires explicit `id`:
```python
from appPublic.uniqueID import getID
await sor.C('rolepermission', {'id': getID(), 'roleid': role_id, 'permid': perm_id})
```

Use raw SQL for users table updates (sor.U adds wrong WHERE clause):
```python
await sor.sqlExe("UPDATE users SET passwd = ${pw}$ WHERE id = ${id}$", {'pw': encoded, 'id': uid})
```

DBPools initialization:
```python
from appPublic.jsonConfig import getConfig
from sqlor.dbpools import DBPools
config = getConfig('.')
DBPools(config.databases)
```

### 16. permission Table Unique-on-Path vs rolepermission Multi-Role (CRITICAL)

**Symptom**: Trying to insert `permission` rows for the same path with different roles hits `IntegrityError: Duplicate entry '/tenant/xxx' for key 'permission_idx1'`.

**Root cause**: The `permission` table has a unique index on `path` alone — NOT on `(path, permtype)`. Each path can only have ONE `permission` row. To grant multiple roles access to the same path, add multiple `rolepermission` rows (one per role) all pointing to the SAME `permission.id`.

**WRONG** — trying to insert duplicate permission paths:
```python
# ❌ permission_idx1 unique on path → 2nd insert fails!
for role in ['logined', 'owner.superuser']:
    await sor.C('permission', {'id': getID(), 'path': '/tenant/', 'name': '/tenant/'})
```

**CORRECT** — one permission row, multiple rolepermission rows:
```python
# 1. Ensure permission exists (idempotent)
recs = await sor.sqlExe('SELECT id FROM permission WHERE path=${path}$', {'path': path})
perm_id = recs[0].id if recs else await create_permission(path)

# 2. Grant to multiple roles via rolepermission
for role_id in [logined_id, reseller_admin_id, owner_super_id]:
    existing = await sor.sqlExe(
        'SELECT id FROM rolepermission WHERE roleid=${rid}$ AND permid=${pid}$',
        {'rid': role_id, 'pid': perm_id})
    if not existing:
        await sor.sqlExe(
            'INSERT INTO rolepermission (id, roleid, permid) VALUES (${id}$, ${rid}$, ${pid}$)',
            {'id': getID(), 'rid': role_id, 'pid': perm_id})
```

**How to diagnose which roles have access**:
```sql
-- See which roles have access to a path
SELECT r.orgtypeid, r.name, p.path
FROM rolepermission rp
JOIN permission p ON p.id = rp.permid
JOIN role r ON r.id = rp.roleid
WHERE p.path LIKE '/tenant%' ORDER BY p.path;

-- Find permission entries with NO rolepermission links (unreachable paths)
SELECT p.path FROM permission p
LEFT JOIN rolepermission rp ON rp.permid = p.id
WHERE p.path LIKE '/module%' AND rp.permid IS NULL;
```

**Complete load_path.py pattern**: See `references/tenant-load-path-db-direct.md` for the full DB-direct script used in the tenant module fix.

### 16b. load_path.py Idempotency Trap — Stale rolepermission Links (CRITICAL)

**Symptom**: `load_path.py` runs and reports `0 path inserted`. Permission rows exist in DB. But curl returns 401 Unauthorized. The path has rolepermission links to specific role IDs instead of `'any'`.

**Root Cause**: `load_path.py` is fully idempotent — `if recs: continue` skips paths that already have a `permission` row. If the path was first registered with the WRONG roles (e.g., specific orgtypeid.name roles instead of `any`), re-running the script does NOTHING because the permission row already exists. The stale rolepermission links remain.

**How this happens**: A path first gets registered via a different script (e.g., `init_superuser.py` or a module's per-module `load_path.py`) with specific roles. Later, the Sage root `load_path.py` declares it as `any`, but the permission row already exists so it's skipped entirely.

**Detection**:
```sql
-- Check which roles are linked to a permission that should be 'any'
SELECT rp.roleid FROM rolepermission rp
JOIN permission p ON p.id = rp.permid
WHERE p.path = '/tenant/index.ui';
-- If roleid shows specific IDs (e.g. '2xzl8bx6ie...') instead of 'any' → stale links
```

**Fix** — Delete stale rolepermissions, insert correct 'any' link:
```python
from appPublic.uniqueID import getID
for path in ['/tenant', '/tenant/index.ui']:
    recs = await sor.R('permission', {'path': path})
    permid = recs[0].id
    old_rps = await sor.R('rolepermission', {'permid': permid})
    for rp in old_rps:
        await sor.D('rolepermission', {'id': rp.id})
    await sor.C('rolepermission', {'id': getID(), 'roleid': 'any', 'permid': permid})
```

**Prevention**: When adding new `any` paths to `load_path.py`, check for stale rolepermission before running:
```sql
SELECT rp.roleid FROM rolepermission rp JOIN permission p ON p.id = rp.permid
WHERE p.path = '/new/path';
-- If non-empty and roleid != 'any', manually delete first
```

### 17. Common RBAC 403 Diagnosis Workflow

When a user reports 403 on a path that should be accessible:

```sql
-- 1. Does the permission row exist?
SELECT * FROM permission WHERE path = '/the/exact/path';

-- 2. Which roles are linked via rolepermission?
SELECT r.orgtypeid, r.name FROM rolepermission rp
JOIN role r ON r.id = rp.roleid
JOIN permission p ON p.id = rp.permid
WHERE p.path = '/the/exact/path';

-- 3. What roles does the user have?
SELECT r.orgtypeid, r.name FROM userrole ur
JOIN role r ON r.id = ur.roleid
WHERE ur.userid = '<userid>';

-- 4. Cross-reference: user's role keys (e.g. reseller.admin) must appear in step 2.
--    Note: get_userroles() auto-expands to *.name and orgtypeid.* wildcards,
--    so a user with reseller.admin also matches *.admin and reseller.*.
```

**RBAC cache**: `UserPermissions.rp_caches` has 600s TTL. After DB changes, either restart Sage or wait for cache expiry.

### 18. Allow Same Phone Number for Multiple User Registrations

`sms_register.dspy` blocks registration if phone exists. Remove the `sor.R('users', {'mobile': mobile})` check. `code_login.dspy` and `phone_login.dspy` already handle multi-user-per-phone via selection list UI.

### 19. get_search_roleid.dspy — Return ALL Roles (Not Org-Type Filtered)

**Problem**: `get_search_roleid.dspy` filtered by current user's `org_type`. The `code` uitype in list view resolves stored `roleid` values — if a role belongs to a different org_type, it's missing from filtered results and displays as raw ID.

**Fix**: Remove org_type filter, return all non-system roles unconditionally:
```sql
SELECT id, CONCAT(orgtypeid, '.', name) as name FROM role
WHERE id NOT IN ('anonymous', 'any', 'logined')
AND orgtypeid != '*' AND name != '*'
AND orgtypeid IS NOT NULL AND name IS NOT NULL
ORDER BY orgtypeid, name
```

### permission.id VARCHAR(32) Truncation

**Problem**: `permission.id` is generated as `perm_` + path with `/` replaced by `_`. Long paths like `/knowledge_bases_list/upload.dspy` → `perm__knowledge_bases_list_upload.dspy` (39 chars) exceed the `VARCHAR(32)` column limit.

**Consequence**: MySQL silently truncates the ID to 32 chars (`perm__knowledge_bases_list_uploa`). `rolepermission.permid` is also truncated. Since both are truncated identically, the JOIN still works — but the warning `Data truncated for column 'id'` appears in logs. Two different long paths could theoretically collide but in practice this is rare.

**Fix**: Not strictly necessary for correctness (truncation is consistent), but can be avoided by using shorter path names or manually crafting shorter permission IDs.

## Verification Commands

```bash
# Check perm_config.py has no /main prefix
grep '/main/' perm_config.py  # Should return nothing

# Test login + permission
curl -c /tmp/cookie.txt -X POST http://localhost:8080/rbac/user/up_login.dspy \
  -d 'username=super&password=Kyy@123456'
curl -b /tmp/cookie.txt http://localhost:8080/customer_management/customer_list.ui
```

## Admin User Creation
- **Role-based tier analysis**: See `references/harnessed-permission-analysis.md` for a complete walkthrough of how to analyze module business nature and classify paths into Public/Logined/Admin/Superuser tiers.
- **Sage deployment patterns**: See `references/sage-deployment-patterns.md` for module installation, integration, four-tier permission model, role definitions, and permission init script template.
- **Module-level RBAC script template**: See `references/per-module-load-path.md` for the `scripts/load_path.py` Python template — direct DB operations, idempotent path registration, role assignment guide.
- **Admin user creation**: See `references/admin-user-creation.md` for creating superuser accounts with `password_encode()` bcrypt hashing, org creation, and role assignment.
- **User self-registration flow**: See `references/user-registration-flow.md` for the `register_user` function, three registration entry points, role customization during registration, and profile editing form patterns.
- **Transfer payment email scanning**: See `references/transfer-payment-patterns.md` for POP3 email parsing, regex patterns, DSPY→Python delegation, and common pitfalls.
- **ahserver auth middleware**: See `references/ahserver-auth-middleware.md` for the full middleware architecture, how all requests (including static files) pass through RBAC, common failure modes (500 on static resources, DB connection failures, cache miss slowdowns), and cache details.
- **WebSocket debugging**: See `references/websocket-debugging.md` for WSS architecture, three path contexts, frontend/ahserver pitfalls, and deployment checklist.
- **RBAC init script**: See `references/rbac-init-script.md` for a complete standalone Python script that creates `any`/`logined` roles and registers permissions — useful for independent apps without Sage's `set_role_perm.py`.