yumoqing/pipeline_core
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity
pipeline_core/skills_library/all/rbac-permission-initialization-pattern/SKILL.md

45 KiB
Raw Blame History

name description author tags
rbac-permission-initialization-pattern Pattern for initializing RBAC permissions in business modules that use wildcard expansion, role ID matching, and path registration Hermes Agent
rbac
permissions
init
multi-tenant
sage

RBAC Permission Initialization Pattern

Overview

When deploying a business module with RBAC authentication, the permission initialization must handle several complex scenarios: path normalization, role wildcard expansion, CRUD file structure, and URL rewriting edge cases (WSS, index auto-match).

User preference: Each business module owns its own scripts/load_path.py that registers its paths directly via Sage DB operations (mirroring Sage's load_path.py internals). This keeps permissions self-contained per module and runnable from any Sage environment. See references/per-module-load-path.md for the Python template pattern. The Sage-level load_path.py is the canonical declarative source of truth, but for per-module workflows, the module's own script is preferred.

Role-Based Permission Analysis Methodology

Before writing permission scripts, analyze each role's responsibilities and classify paths into tiers:

Step 1: Document role职责

owner.superuser    — 系统级: 机构类型/角色/权限管理, 添加业主管理员
*.admin            — 机构级: 添加本机构人员, 分配人员角色
reseller.operator  — 运营: 产品管理/供应商合同/定价/统一折扣/营销
reseller.sale      — 销售: 客户管理/客户特殊折扣
reseller.accountant — 财务: 线下充值/对账结算
reseller.maintainer — 运维维护
customer.customer  — 终端客户用户
logined            — 所有已登录用户

Step 2: Analyze module business nature

  • Ask: Is this a domain-specific business module (CRM, accounting) or a general tool service (AI agent, reasoning)?
  • General tool services → broader access (all logined users can use)
  • Domain-specific modules → restricted to relevant roles

Step 3: Classify into permission tiers

Tier Role set Path types
Public any 登录/注册/认证页面、静态资源(img/css)
Logined all 登录角色 用户自助服务(个人信息、API Key)、数据查看(列表+get)、用户自己的CRUD(通过user_id隔离)
Admin superuser + *.admin 系统/机构配置管理、用户管理、机构管理
Superuser owner.superuser only 全局元数据(角色/权限/机构类型)、高危操作(技能部署)

Step 4: Register CRUD paths comprehensively

  • JSON CRUD alias → directory {alias}/ with index.ui, get_*.dspy, add_*.dspy, update_*.dspy, delete_*.dspy
  • Custom api/ directory may also contain CRUD endpoints — register both
  • Every CRUD directory needs TWO paths (see Pitfall 2)

Key Principles

1. Wildcard Expansion (Filesystem Scanning)

Problem: rbac.check_roles_path() does exact matching. Permission definitions must be registered as concrete URLs in the DB.

CRITICAL: Scan ALL file types — .ui, .dspy, .js, .css. RBAC protects all static resources.

IMPORTANT: ahserver auto-serves .css and .js files from module wwwroot/ directories — they are injected into HTML responses without explicit <link>/<script> tags in .ui files. Do NOT manually reference these files in .ui widgets. However, they are still subject to RBAC checks. Every .css/.js file must be registered in the permission system. Theme files needed before auth should use any role.

Symlink handling: Use followlinks=True in os.walk, but guard against self-referencing symlinks (e.g., main -> wwwroot):

real_wwwroot = os.path.realpath(wwwroot_path)
visited_real = set()
for root, dirs, files in os.walk(wwwroot_path, followlinks=True):
    real_root = os.path.realpath(root)
    if real_root in visited_real:
        dirs.clear()
        continue
    visited_real.add(real_root)
    dirs[:] = [d for d in dirs if os.path.realpath(os.path.join(root, d)) != real_wwwroot]

2. Directory URL Dual Path Registration (including root /)

Problem: User visits a directory URL and gets 401, even though the index.ui inside has permission.

Root Cause: ahserver's website.indexes config auto-matches index.ui for directory URLs. RBAC checks the raw request path, NOT the resolved file path. When the user visits /harnessed_agent/hermes_memory, RBAC sees /harnessed_agent/hermes_memory (NO /index.ui suffix). Same applies to root: visiting http://host/ checks path /, NOT /index.ui.

Fix: Register BOTH the directory path AND the index file path:

# CRUD subdirectory — register both:
python set_role_perm.py "logined" "/harnessed_agent/hermes_memory"
python set_role_perm.py "logined" "/harnessed_agent/hermes_memory/index.ui"

# ROOT path — same pattern, register both:
python set_role_perm.py "any" "/"
python set_role_perm.py "any" "/index.ui"

Pitfall for auto-discovery scripts: When scanning wwwroot files to auto-register permissions (e.g., init_any_permissions.py), always explicitly append / to the path list — it's not a real file so os.listdir() won't find it:

root_paths = ["/" + f for f in os.listdir(wwwroot_dir) if ...]
root_paths.append("/")  # CRITICAL: root path is not a file

3. Path Registration (NO /main prefix)

CRITICAL: Do NOT add /main prefix to permission paths. The config.json paths mapping handles URL routing at the HTTP level. RBAC permission checks use paths relative to wwwroot.

def normalize_path(path):
    if path.startswith('/main'):
        cleaned = path[5:]
        return cleaned if cleaned else '/'
    return path

4. WSS WebSocket Path Handling

CRITICAL: Server logs confirm RBAC checks the FULL path INCLUDING /wss/ prefix. Both UI and RBAC use the same path format.

Verified from Sage server logs: path='/wss/harnessed_reasoning/reasoning_console.wss' permission check failed

Rule:

  • UI reference: {{entire_url('/wss/module/file.wss')}} — full path with /wss/
  • RBAC permission: /wss/module/file.wss — SAME path, WITH /wss/ prefix
  • set_role_perm.py: python set_role_perm.py "logined" "/wss/module/file.wss"

5. ahserver .wss Processor Mapping

Symptom: WebSocket to .wss endpoint returns 401 or is handled as a static file. Log: "handle as a normal file". Root cause: conf/config.json processors list only has [".ws","ws"], not [".wss","ws"]. Fix: Add [".wss","ws"] to the processors array in conf/config.json:

"processors":[
    [".ws","ws"],
    [".wss","ws"],
    ...
]

6. RBAC 'any' and 'logined' Roles Require role Table Entries

Symptom: Users get 401 despite rolepermission having correct entries. Root cause: userperm.py joins rolepermission + permission + role tables. Missing role rows mean no results for those keys.

Critical roles that MUST exist in role table:

-- Anonymous users (public resources)
INSERT INTO role (id, orgtypeid, name) VALUES ('any', '*', 'any');

-- Authenticated users (module access after login)
INSERT INTO role (id, orgtypeid, name) VALUES ('logined', '*', 'logined');

Why logined is critical: userperm.py automatically assigns ['any', 'logined'] to all authenticated users. If the logined role doesn't exist in the role table, logged-in users have NO permissions despite having valid sessions.

Detection: If login succeeds but all module paths return 401:

-- Check if logined role exists
SELECT * FROM role WHERE name='logined';
-- If empty, create it and register paths

Fix: Ensure both roles exist and paths are registered in load_path.py:

PATHS_ANY = ["/", "/index.ui", "/menu.ui", "/bricks/**", "/rbac/user/login.ui"]
PATHS_LOGINED = ["/module/**", "/module/api/%", "/module/index.ui", ...]

7. role Table Missing name Column

Symptom: Unknown column 'c.name' in 'SELECT' when RBAC loads role permissions. Root cause: DDL from xlsx may not include name column that userperm.py expects. Fix: ALTER TABLE role ADD COLUMN name VARCHAR(100); then populate from role field.

8. Role Wildcard Matching (orgtypeid='*' 通配所有机构)

get_userroles() expands roles into THREE keys:

roles.append(f'{r.orgtypeid}.{r.name}')   # e.g., 'customer.sales_rep'
roles.append(f'{r.orgtypeid}.*')           # e.g., 'customer.*'
roles.append(f'*.{r.name}')                # e.g., '*.sales_rep'

Create roles with orgtypeid='*' — RBAC matches across all organizations via *.{name}.

9. Core Framework UI Files Must Be any Role

Symptom: Visiting the root path / or index.ui returns HTTP 401, even though login works and module permissions are set. The app appears completely inaccessible.

Root Cause: Sage's root page loads several core framework files (top.ui, center.ui, bottom.ui, bricks CSS/JS, public assets). If these are not registered for the any role, anonymous users cannot load the shell UI — even though they could theoretically log in, the framework files they need to see the login page are blocked.

Required any permissions for every Sage deployment:

# Core UI shell (the app frame)
python set_role_perm.py "any" "/"
python set_role_perm.py "any" "/index.ui"
python set_role_perm.py "any" "/top.ui"
python set_role_perm.py "any" "/center.ui"
python set_role_perm.py "any" "/bottom.ui"

# Bricks framework (ALL files recursively — CSS, JS, imgs, 3parties, examples, docs)
find wwwroot/bricks -type f | while read f; do
    rel="${f#wwwroot}"
    python set_role_perm.py "any" "$rel"
done

# Public assets (ALL files recursively)
find wwwroot/public -type f | while read f; do
    rel="${f#wwwroot}"
    python set_role_perm.py "any" "$rel"
done

# Common navigation/menu files
python set_role_perm.py "any" "/user_menu.ui"
python set_role_perm.py "any" "/menu.ui"
python set_role_perm.py "any" "/accordion.ui"
python set_role_perm.py "any" "/app_panel.ui"

Verification: After setting permissions, restart the app and test:

curl -s -o /dev/null -w '%{http_code}' http://localhost:9180/              # 200
curl -s -o /dev/null -w '%{http_code}' http://localhost:9180/bricks/css/bricks.css  # 200
curl -s -o /dev/null -w '%{http_code}' http://localhost:9180/bricks/imgs/app.svg     # 200
curl -s -o /dev/null -w '%{http_code}' http://localhost:9180/public/index.ui         # 200

Important: Module-level wwwroot directories that live OUTSIDE the sage wwwroot (e.g., harnessed_agent, harnessed_reasoning in ~/repos/) must be symlinked into sage/wwwroot/ so ahserver can serve them:

cd ~/repos/sage/wwwroot
ln -s ../../harnessed_agent/wwwroot harnessed_agent
ln -s ../../harnessed_reasoning/wwwroot harnessed_reasoning

12. RBAC Tools Placement and .dspy Function Pattern

User correction: RBAC utility tools (path-permission queries, unauth file scans, etc.) MUST live in the rbac module (~/repos/rbac/wwwroot/), NOT in sage/scripts/ or any other module's scripts directory.

CRITICAL RULE: .dspy files MUST NOT contain import statements or f-strings with braces in dict literals.

import is prohibited because .dspy runs in a restricted exec() namespace. All logic with imports must live in the module's Python files.

f-string braces {var} inside dict literals cause SyntaxError: '{' was never closed. The exec() environment misparses {e} inside f"..." as a dict brace. Use string concatenation instead:

# WRONG — SyntaxError in .dspy
return {"widgettype": "Message", "options": {"message": f"处理失败: {e}"}}

# CORRECT — use concatenation
return {"widgettype": "Message", "options": {"message": "处理失败: " + str(e)}}

The debug(f'...') / exception(f'...') calls are fine since they don't nest inside dict braces.

This includes import os, import json, from collections import defaultdict, from appPublic.dictObject import DictObject, etc. The .dspy file is executed in an inline eval context with a limited namespace — imports are prohibited. All logic with imports must live in the module's Python files.

Correct pattern for RBAC utility tools (and any non-trivial .dspy):

  1. rbac/<module>/tools.py — Python module with imports and all business logic. Functions return (title, message, is_error) tuples.
  2. rbac/<module>/init.py — Register functions on ServerEnv in load_rbac(): 
    from .tools import query_path_roles, scan_unauth_files
# In load_rbac():
env.query_path_roles = query_path_roles
env.scan_unauth_files = scan_unauth_files
  3. wwwroot/<tool>.ui — ModalForm UI for user input
  4. wwwroot/<tool>.dspy — Zero imports. Call module functions via request._run_ns: 
    async with get_sor_context(request._run_ns, 'rbac') as sor:
    title, message, is_error = await request._run_ns.query_path_roles(sor, path)
if is_error:
    return UiError(title=title, message=message)
return UiMessage(title=title, message=message)
  5. wwwroot/admin_menu.ui — Add menu entry (accessed via /appbase/menu.ui for superuser)
  6. Register permissions — Both .ui and .dspy paths need owner.superuser role

Function return convention: Module functions should return (title, message, is_error) tuples. The .dspy file unpacks these and decides between UiError and UiMessage. This keeps HTML rendering logic in the Python module where imports are allowed.

Standalone .py → .dspy conversion rules:

  • Input: replace sys.argv → params_kw.get('name', '')
  • DB context: replace DBPools(config.databases) + sqlorContext('sage') → get_sor_context(request._run_ns, 'rbac')
  • Output: replace print() → return UiMessage(title='...', message='...') or UiError
  • sor.sqlExe() always requires ns dict, even with zero placeholders: sqlExe("SELECT ...", {}) — passing without {} raises TypeError: missing 1 required positional argument: 'ns'

DSPY→Python delegation pattern: When a .dspy needs complex logic, put it in a Python method on the provider/class, accessible via env.PROVIDERS:

  1. Add the method to the provider's .py file — does the work, returns (title, message)
  2. Ensure the provider is registered on env.PROVIDERS (usually already done in load_xxx())
  3. .dspy just calls: provider = env.PROVIDERS.get('xxx'); title, msg = await provider.method(params, env)

This keeps the .dspy thin (no imports, no complex logic, no f-strings in dicts) while Python code has full import access.

DSPY→Python delegation pattern: When a .dspy needs complex logic (email scanning, file I/O, API calls), put the logic in a Python method and register it on ServerEnv. The .dspy just calls the method:

9a. Cron DSPY Endpoints: any Role + IP Whitelist Pattern

Pattern: External crontab triggers .dspy endpoints via curl localhost. Since cron has no auth token, these endpoints MUST use any role in RBAC. Security is enforced by IP whitelist in the Python function, NOT by RBAC.

Architecture:

crontab → curl localhost → .dspy (any role) → init.py function (IP check)

load_path.py — register cron paths as any:

PATHS_ANY = [
    f"/{MOD}/cron/etl_sync.dspy",
    f"/{MOD}/cron/etl_aggregate.dspy",
    f"/{MOD}/cron/etl_provider_cost.dspy",
]

init.py — registered function with IP whitelist:

async def cron_etl_sync(request):
    ip = request['client_ip']
    if ip not in ['127.0.0.1']:
        return {'error': 'IP ' + ip + ' not allowed'}
    env = request._run_ns
    async with get_sor_context(env, MODULE_NAME) as sor:
        count = await sync_call_fact(sor)
        return {'status': 'ok', 'synced': count}

# In load_module():
env.cron_etl_sync = cron_etl_sync

.dspy thin wrapper (NO imports, NO f-strings in dicts):

# ETL: description of what this does
result = await request._run_ns.cron_etl_sync(request)
return str(result)

build.sh crontab entry (guard against duplicates):

dm_cron="*/5 * * * * curl -s http://localhost:9180/${MOD}/cron/etl_sync.dspy"
if ! echo "$cc"|grep -Fq "etl_sync"; then
    (echo "$cc";echo "$dm_cron") | crontab -
fi

Pitfall: Never put import or from in cron .dspy files — the exec() namespace prohibits it. All imports and logic must live in init.py registered functions.

Pitfall: Cron endpoints use any role (not logined) because curl from crontab has no session cookie. The IP whitelist (request['client_ip']) in the Python function is the security boundary.

9b. Login DSPY Endpoints Must Be any Role (CRITICAL)

Symptom: Login page loads (200) but submitting login returns 401 Unauthorized. User sees "401: Unauthorized" text instead of being redirected to the app.

Root Cause: The login UI pages (login.ui, userpassword_login.ui) are registered as any, but the dspy endpoints that process the login POST requests are NOT. RBAC blocks the POST before authentication can happen.

Required any permissions for ALL login-related dspy endpoints:

PATHS_ANY = [
    # Login UI pages (already known)
    "/rbac/user/login.ui",
    "/rbac/user/register.ui",
    "/rbac/userpassword_login.ui",
    # Login DSPY endpoints (MUST ALSO be any!)
    "/rbac/userpassword_login.dspy",     # password login POST handler
    "/rbac/phone_login.dspy",            # phone login POST handler
    "/rbac/user/code_login.dspy",        # SMS code login POST handler
    "/rbac/gen_sms_code.dspy",           # send SMS verification code
    "/rbac/user/register.dspy",          # user registration POST handler
    "/rbac/user/sms_register.dspy",      # SMS registration POST handler
    "/rbac/user/up_login.dspy",          # session-based login
    "/rbac/user/logout.dspy",            # logout
]

Detection: curl -s -o /dev/null -w "%{http_code}" http://host/rbac/userpassword_login.dspy returns 401 → missing any permission.

Fix: Register all login dspy paths as any in load_path.py or RBAC init script.

9c. Independent App set_role_perm.py Wrapper

Context: Independent apps (pipeline-app, etc.) don't have Sage's root-level set_role_perm.py. Module load_path.py scripts call py3/bin/python set_role_perm.py <role> <path> which expects the script at app root.

Solution: Create set_role_perm.py at app root as a thin wrapper:

#!/usr/bin/env python3
"""Wrapper for RBAC permission registration. Called by modules' load_path.py."""
import sys, os, asyncio
sys.path.insert(0, os.path.dirname(os.path.abspath(__file__)))
from rbac.set_role_perms import set_role_perm as _set_role_perm
from appPublic.jsonConfig import getConfig
from sqlor.dbpools import DBPools

async def main(role, path):
    parts = path.strip('/').split('/')
    module = parts[0] if parts else 'app'
    config = getConfig('.', {'workdir': '.'})
    db = DBPools(config.databases)
    await _set_role_perm('APP_DBNAME', module, '*', role, path)

def run(coro):
    loop = asyncio.new_event_loop()
    asyncio.set_event_loop(loop)
    try: loop.run_until_complete(coro)
    finally: loop.close()

if __name__ == '__main__':
    if len(sys.argv) != 3:
        print(f"Usage: {sys.argv[0]} <role> <path>"); sys.exit(1)
    run(main(sys.argv[1], sys.argv[2]))

Load_path.py universal pattern (works for any app, not just Sage):

def register_paths():
    for path in PATHS_ANY:
        subprocess.run(["py3/bin/python", "set_role_perm.py", "any", path])
    for path in PATHS_LOGINED:
        subprocess.run(["py3/bin/python", "set_role_perm.py", "logined", path])

Pitfall: Do NOT hardcode Sage paths or SAGE_ROOT detection in load_path.py. The script runs from the app's root directory using the app's own venv.

9d. Async Startup Tasks MUST Use add_startup, NOT asyncio.ensure_future

Symptom: Task was destroyed but it is pending! coroutine 'X.run' was never awaited on startup.

Root Cause: Module init() functions run SYNCHRONOUSLY before the event loop starts (called from configuredServer.py). asyncio.ensure_future() or asyncio.create_task() in init phase creates tasks that are never scheduled.

Fix: Use add_startup(coro) from ahserver.configuredServer — it registers a callback that runs once when the event loop starts:

from ahserver.configuredServer import add_startup

def load_module():
    for name in providers:
        PROVIDERS[name] = get_provider(name, conf)
        if name == 'transfer':
            add_startup(PROVIDERS[name].run)  # ✓ runs after event loop starts
            # NOT: asyncio.ensure_future(PROVIDERS[name].run())  # ✗ init phase, no event loop

10. Permission Cache Refresh

RBAC caches permissions in memory. Must restart the app after DB changes.

IMPORTANT: ALL requests pass through RBAC middleware — even static files (.css, .js, .png, .svg). There is no static file bypass. If the RBAC database is unreachable, every single request returns 500, including all static resources. See references/ahserver-auth-middleware.md for the full middleware architecture, common failure modes, and cache details.

10b. Script Output Silencing for Large Batches

Symptom: set_role_perm.py scripts with hundreds of paths time out or flood the terminal. Root cause: Each permission registration prints output (e.g., logined, xxxxx perm add or already exists). With 1500+ permission entries, the output buffer overflows. Fix: Redirect output in the set_perm function:

set_perm() {
    local role="$1"
    local path="$2"
    python set_role_perm.py "${role}" "${path}" > /dev/null 2>&1
    COUNT=$((COUNT + 1))
}

10c. set_core_perms.sh Pattern for Static Assets

Sage uses set_core_perms.sh to batch-register any role permissions for static assets (bricks, public, core UI). The script uses a helper function and find:

COUNT=0
set_perm() {
    python set_role_perm.py "$1" "$2" > /dev/null 2>&1
    COUNT=$((COUNT + 1))
}
for p in "/index.ui" "/top.ui" "/center.ui" "/bottom.ui"; do
    set_perm "any" "$p"
done
while IFS= read -r file; do
    rel="${file#/home/hermesai/repos/sage/wwwroot}"
    set_perm "any" "$rel"
done < <(find wwwroot/bricks -type f 2>/dev/null)

10d. Wildcard % and ** as First-Class Permission Standards

User correction: Manually listing every .dspy file in load_path.py leads to 403 errors when new files are added but the script isn't updated. Use % or ** wildcards to auto-cover directories.

Two wildcard suffixes are supported by userperm.check_roles_path():

  • % suffix — SQL LIKE-style wildcard (e.g., /module/api/% matches /module/api/anything)
  • ** suffix — glob-style wildcard (e.g., /module/** matches /module/anything)

Both produce identical behavior: prefix match against the request path. Use whichever style you prefer, but be consistent within a project.

Standard patterns in load_path.py:

# api/ directory — covers all .dspy endpoints
f"/{MOD}/api/%",

# CRUD subdirectories — covers add/get/update/delete .dspy + index.ui
f"/{MOD}/llm/%",
f"/{MOD}/llmcatelog/%",

# Loose top-level .dspy files (non-api, non-CRUD)
f"/{MOD}/%.dspy",

# Static assets
f"/{MOD}/imgs/%",

When to list files explicitly vs. wildcards:

  • ✅ Wildcard: directories with many files that grow over time (api/, CRUD dirs, imgs/)
  • ✅ Wildcard: loose .dspy at top level (/{MOD}/%.dspy)
  • ✅ Explicit: specific .ui pages that need role-based access (e.g., admin-only pages)
  • ✅ Explicit: callback endpoints that need any role (e.g., api/rl_callback.dspy)

Role-based separation example (reallife_asset pattern):

# logined — customer-visible pages + all APIs (scripts do get_user() checks internally)
PATHS_LOGINED = [
    f"/{MOD}",
    f"/{MOD}/index.ui",
    f"/{MOD}/create_validate.ui",    # customer page
    f"/{MOD}/upload_asset.ui",       # customer page
    f"/{MOD}/api/%",                 # all API endpoints
]

# operator — admin-only pages (separate list for specific roles)
PATHS_OPERATOR = [
    f"/{MOD}/group_manage.ui",
    f"/{MOD}/vendor_config_manage.ui",
    f"/{MOD}/asset_manage.ui",
]

10e. Wildcard SQL LIKE for Bulk Permission Checks

To check or register permissions using SQL LIKE patterns, use set_role_perm.py with % wildcard:

# Register all /bricks/* paths (matches existing records via LIKE)
./py3/bin/python set_role_perm.py any /bricks/%

Note: set_role_perm.py internally uses path LIKE ${path}$ SQL when % is present in the path argument. Only matches paths already in the permission table — use find + loop for newly-added files.

11b. sage/wwwroot/menu.ui is gitignored

Sage's .gitignore includes wwwroot/ because most files are symlinks from module repos. The main wwwroot/menu.ui is a regular file but is still ignored. Changes to it must be committed with git add -f wwwroot/menu.ui or kept local. RBAC module's own admin_menu.ui lives in ~/repos/rbac/wwwroot/admin_menu.ui and IS tracked by git.

11c. Standalone Superuser Init Scripts

For modules that need a quick way to grant owner.superuser all permissions (e.g., after adding new endpoints or during initial setup), create a standalone init_superuser.py at the module root.

CRITICAL: py path must come from sage_root, NOT app_root. Multi-module repos (like CMS) often lack their own py3 virtualenv — using app_root's python will fail with FileNotFoundError. Always construct: py = os.path.join(sage_root, "py3", "bin", "python").

Pattern:

# init_superuser.py — grant owner.superuser all module permissions
import os, sys, subprocess

sage_root = None
for c in [os.path.expanduser("~/repos/sage"), os.path.expanduser("~/sage")]:
    if os.path.isdir(os.path.join(c, "py3", "bin")):
        sage_root = c
        break
if not sage_root:
    print("ERROR: Sage not found"); sys.exit(1)

py = os.path.join(sage_root, "py3", "bin", "python")  # MUST be sage_root, not app_root
sp = os.path.join(sage_root, "set_role_perm.py")

def run(role, paths):
    for p in paths:
        subprocess.run([py, sp, role, p], cwd=sage_root, capture_output=True)

superuser_paths = [
    "/module/index.ui",
    "/module/api/%",
    # ... all paths
]

run("owner.superuser", superuser_paths)

When to use:

  • After manually adding new endpoints to production and need to grant superuser access
  • Initial module setup when superuser needs full access before other roles are configured
  • Quick permission refresh for superuser after schema changes

Difference from load_path.py:

  • load_path.py defines all roles and their permissions (any, logined, operator, etc.)
  • init_superuser.py is a convenience script specifically for granting superuser all paths
  • Both use set_role_perm.py but serve different purposes

Pitfall: Naming conflict with scripts/init_superuser.py Some repos have TWO scripts with similar names:

  • scripts/init_superuser.py — creates a superuser account in the database (INSERT INTO users)
  • Root init_superuser.py — grants owner.superuser role all RBAC paths

This causes confusion and build.sh may reference the wrong one. Fix: Rename the root-level RBAC script to init_superuser_permissions.py to disambiguate. Always use distinct names for scripts that do different things (user creation vs. permission assignment).

11d. Auto-Discovery any Permission Script (init_any_permissions.py)

For multi-module repos where wwwroot/ directories contain many files that change over time, create an auto-discovery script that scans filesystem and sets any permissions dynamically:

Key design points:

  • Scan each module's wwwroot/ with os.walk(), mapping to correct URL prefix
  • Always explicitly append "/" to root paths — it's not a file so filesystem scanning won't find it, but RBAC checks raw request path / when users visit the root URL
  • Skip symlinks pointing outside the repo (prevents registering other modules' files)
  • Skip __pycache__, .git, .pyc, .bak etc.
  • Handle bricks/ symlink separately (created during build.sh, may not exist)
  • Check symlink targets with os.path.realpath() to detect external links

Multi-module URL prefix mapping:

entcms/wwwroot/*        → /<file>           (no prefix, root-level)
dingdingflow/wwwroot/*  → /dingdingflow/<file>  (with module prefix)
bricks/*                → /bricks/<file>    (framework prefix)

Symlink safety pattern:

if os.path.islink(full_path):
    link_target = os.path.realpath(full_path)
    if not link_target.startswith(app_root):
        continue  # skip external symlinks

11e. CRUD UI Files Generated by build.sh

CRUD management pages (e.g., cms_content_list/index.ui) do NOT exist in the repo — they are generated by xls2ui during build.sh. Before build:

  • json/*.json has CRUD definitions
  • wwwroot/ has hand-written .ui files (index, admin, menu, etc.)

After build:

  • wwwroot/<alias>/index.ui + CRUD .dspy files are generated by xls2ui
  • These generated files also need RBAC permissions (wildcard % handles this)

Pitfall: load_path.py may reference files like products.ui that don't exist yet (referenced but not created). Verify file existence before registering permissions.

12. Bulk Module Scanning for load_path.py

When auditing all repos to find which modules need scripts/load_path.py:

# Step 1: Find modules WITH web content but WITHOUT load_path.py
for d in ~/repos/*/; do
  mod=$(basename "$d")
  mdir="$d/wwwroot"
  [ ! -d "$mdir" ] && continue
  has_ui=$(find "$mdir" -name "*.ui" -type f | head -1)
  has_lp="NO"; [ -f "$d/scripts/load_path.py" ] && has_lp="YES"
  [ -n "$has_ui" ] && [ "$has_lp" = "NO" ] && echo "NEEDS: $mod"
done

# Step 2: For each module, collect all web paths
# Scan wwwroot/ for .ui/.dspy files, json/ for CRUD alias dirs
find ~/repos/MODULE/wwwroot -type f \( -name "*.ui" -o -name "*.dspy" \) | sort
ls ~/repos/MODULE/json/*.json 2>/dev/null  # CRUD alias directory names

Path classification rules:

  • any role: menu.ui, usermenu.ui, admin_menu.ui, login/register/logout pages, password reset
  • logined role: everything else (module entry /mod, all .ui/.dspy files, api/ endpoints, CRUD alias dirs)
  • Infrastructure modules without wwwroot/ do NOT need load_path.py: ahserver, apppublic, bricks-for-python, sqlor, checklang, bench_static
  • Note: appbase, dapi, uapi, msp DO have wwwroot/ and need load_path.py

Template for generating load_path.py: See references/per-module-load-path.md

13. set_role_perm.py Role Name Must Be orgtypeid.name Format (CRITICAL)

Problem: load_path.py passes "developer" to set_role_perm.py, which crashes at line 56: orgtypeid, name = role.split('.') — ValueError: not enough values to unpack.

Root Cause: set_role_perm.py only accepts three special role names (anonymous, any, logined) or the orgtypeid.name format. Any other format crashes silently (subprocess captures stderr).

Wrong:

total += register_role_paths("developer", PATHS_DEVELOPER)
total += register_role_paths("admin", PATHS_ADMIN)

Correct:

total += register_role_paths("owner.developer", PATHS_DEVELOPER)
total += register_role_paths("owner.superuser", PATHS_SUPERUSER)
total += register_role_paths("customer.admin", PATHS_CUSTOMER_ADMIN)

Detection: If load_path.py runs without error output but permissions aren't registered, check subprocess stderr — the split('.') error is captured silently by subprocess.run(capture_output=True).

14. Independent App RBAC Database Mismatch (CRITICAL)

Symptom: Permissions exist in the database (rolepermission + permission tables have correct entries), app is restarted, but all paths return 401 Unauthorized. Log shows: userid=None, path='/' permission check failed.

Root Cause: set_role_perm.py hardcodes db.sqlorContext('sage') — it always writes to the sage database. But independent apps (like CMS) have their own get_module_dbname() that returns a different database (e.g., ocai_cms). The RBAC check reads from the app's database (ocai_cms), but permissions were written to sage.

Write → sage, Read → ocai_cms = MISMATCH

Fix: set_role_perm.py supports the SAGE_RBAC_DB environment variable:

# For independent apps, set the target database:
SAGE_RBAC_DB=ocai_cms py3/bin/python ~/sage/set_role_perm.py any /
SAGE_RBAC_DB=ocai_cms py3/bin/python ~/sage/set_role_perm.py owner.superuser /admin.ui

In init scripts, pass the env var via subprocess:

env = os.environ.copy()
env['SAGE_RBAC_DB'] = 'ocai_cms'  # app's database name
subprocess.run([py, sp, role, p], cwd=sage_root, capture_output=True, env=env)

Default behavior unchanged: When SAGE_RBAC_DB is not set, set_role_perm.py defaults to sage — no impact on existing Sage deployments.

Detection: If permission table has the path but RBAC still returns 401, check which database has the data:

-- In the APP's database (e.g., ocai_cms):
SELECT a.*, b.path FROM rolepermission a, permission b WHERE a.permid = b.id AND b.path='/';
-- If empty here but set_role_perm.py said "already exists" → it was written to sage DB

15. Menu Submenu Must Point to Correct Module

When adding submenu entries in the main menu.ui, ensure each submenu points to its OWN module's menu.ui. Two different top-level menu items pointing to the same submenu file will show identical content, hiding the second module's entries entirely.

// WRONG — "推理" shows same content as "代理":
{"name": "agent",    "submenu": "{{entire_url('/harnessed_agent/menu.ui')}}"},
{"name": "reasoning", "submenu": "{{entire_url('/harnessed_agent/menu.ui')}}"},

// CORRECT — each points to its own module:
{"name": "agent",    "submenu": "{{entire_url('/harnessed_agent/menu.ui')}}"},
{"name": "reasoning", "submenu": "{{entire_url('/harnessed_reasoning/menu.ui')}}"},

Assigning User Roles: Critical Pitfalls

sor.R('role', ...) MUST use orgtypeid + name, NOT a fake role field

The role table model has only three fields: id, orgtypeid, name. There is NO role column.

WRONG — sor.R ignores unknown fields, returns ALL roles, role_recs[0] is random:

# Returns 27 records — first one might be owner.* instead of reseller.admin!
role_recs = await sor.R('role', {'role': 'reseller.admin'})

RIGHT — split into model fields:

role_recs = await sor.R('role', {'orgtypeid': 'reseller', 'name': 'admin'})
# Returns exactly 1 record: reseller.admin

For special pseudo-roles (any, logined, anonymous), use id:

role_recs = await sor.R('role', {'id': 'logined'})

logined and any are auto-added — never manually assign

get_userroles() in userperm.py always prepends ['any', 'logined'] to every authenticated user's role list. Do not insert userrole entries for logined or any — it's redundant at best, and at worst can cause confusion when debugging role issues.

Only assign specific {orgtypeid}.{name} roles like reseller.admin, reseller.operator, etc.

Role dropdown filtering by user org_type

When creating a role dropdown (e.g. for userrole CRUD), filter by the current user's org_type. A reseller admin should only see reseller.* roles, not owner.* or customer.*.

userorgid = await get_userorgid()
org_recs = await sor.R('organization', {'id': userorgid})
org_type = org_recs[0].org_type if org_recs else ''

# Filter to user's org_type only (org=0 sees all)
recs = await sor.sqlExe(
    "SELECT id, CONCAT(orgtypeid, '.', name) as name FROM role "
    "WHERE id NOT IN ('anonymous', 'any', 'logined') "
    "AND orgtypeid = ${ot}$ AND name != '*' AND name IS NOT NULL",
    {'ot': org_type})

Use get_search_roleid.dspy pattern with dataurl override in CRUD JSON alters.

Role Permission Table Operations

The rolepermission table requires explicit id:

from appPublic.uniqueID import getID
await sor.C('rolepermission', {'id': getID(), 'roleid': role_id, 'permid': perm_id})

Use raw SQL for users table updates (sor.U adds wrong WHERE clause):

await sor.sqlExe("UPDATE users SET passwd = ${pw}$ WHERE id = ${id}$", {'pw': encoded, 'id': uid})

DBPools initialization:

from appPublic.jsonConfig import getConfig
from sqlor.dbpools import DBPools
config = getConfig('.')
DBPools(config.databases)

16. permission Table Unique-on-Path vs rolepermission Multi-Role (CRITICAL)

Symptom: Trying to insert permission rows for the same path with different roles hits IntegrityError: Duplicate entry '/tenant/xxx' for key 'permission_idx1'.

Root cause: The permission table has a unique index on path alone — NOT on (path, permtype). Each path can only have ONE permission row. To grant multiple roles access to the same path, add multiple rolepermission rows (one per role) all pointing to the SAME permission.id.

WRONG — trying to insert duplicate permission paths:

# ❌ permission_idx1 unique on path → 2nd insert fails!
for role in ['logined', 'owner.superuser']:
    await sor.C('permission', {'id': getID(), 'path': '/tenant/', 'name': '/tenant/'})

CORRECT — one permission row, multiple rolepermission rows:

# 1. Ensure permission exists (idempotent)
recs = await sor.sqlExe('SELECT id FROM permission WHERE path=${path}$', {'path': path})
perm_id = recs[0].id if recs else await create_permission(path)

# 2. Grant to multiple roles via rolepermission
for role_id in [logined_id, reseller_admin_id, owner_super_id]:
    existing = await sor.sqlExe(
        'SELECT id FROM rolepermission WHERE roleid=${rid}$ AND permid=${pid}$',
        {'rid': role_id, 'pid': perm_id})
    if not existing:
        await sor.sqlExe(
            'INSERT INTO rolepermission (id, roleid, permid) VALUES (${id}$, ${rid}$, ${pid}$)',
            {'id': getID(), 'rid': role_id, 'pid': perm_id})

How to diagnose which roles have access:

-- See which roles have access to a path
SELECT r.orgtypeid, r.name, p.path
FROM rolepermission rp
JOIN permission p ON p.id = rp.permid
JOIN role r ON r.id = rp.roleid
WHERE p.path LIKE '/tenant%' ORDER BY p.path;

-- Find permission entries with NO rolepermission links (unreachable paths)
SELECT p.path FROM permission p
LEFT JOIN rolepermission rp ON rp.permid = p.id
WHERE p.path LIKE '/module%' AND rp.permid IS NULL;

Complete load_path.py pattern: See references/tenant-load-path-db-direct.md for the full DB-direct script used in the tenant module fix.

16b. load_path.py Idempotency Trap — Stale rolepermission Links (CRITICAL)

Symptom: load_path.py runs and reports 0 path inserted. Permission rows exist in DB. But curl returns 401 Unauthorized. The path has rolepermission links to specific role IDs instead of 'any'.

Root Cause: load_path.py is fully idempotent — if recs: continue skips paths that already have a permission row. If the path was first registered with the WRONG roles (e.g., specific orgtypeid.name roles instead of any), re-running the script does NOTHING because the permission row already exists. The stale rolepermission links remain.

How this happens: A path first gets registered via a different script (e.g., init_superuser.py or a module's per-module load_path.py) with specific roles. Later, the Sage root load_path.py declares it as any, but the permission row already exists so it's skipped entirely.

Detection:

-- Check which roles are linked to a permission that should be 'any'
SELECT rp.roleid FROM rolepermission rp
JOIN permission p ON p.id = rp.permid
WHERE p.path = '/tenant/index.ui';
-- If roleid shows specific IDs (e.g. '2xzl8bx6ie...') instead of 'any' → stale links

Fix — Delete stale rolepermissions, insert correct 'any' link:

from appPublic.uniqueID import getID
for path in ['/tenant', '/tenant/index.ui']:
    recs = await sor.R('permission', {'path': path})
    permid = recs[0].id
    old_rps = await sor.R('rolepermission', {'permid': permid})
    for rp in old_rps:
        await sor.D('rolepermission', {'id': rp.id})
    await sor.C('rolepermission', {'id': getID(), 'roleid': 'any', 'permid': permid})

Prevention: When adding new any paths to load_path.py, check for stale rolepermission before running:

SELECT rp.roleid FROM rolepermission rp JOIN permission p ON p.id = rp.permid
WHERE p.path = '/new/path';
-- If non-empty and roleid != 'any', manually delete first

17. Common RBAC 403 Diagnosis Workflow

When a user reports 403 on a path that should be accessible:

-- 1. Does the permission row exist?
SELECT * FROM permission WHERE path = '/the/exact/path';

-- 2. Which roles are linked via rolepermission?
SELECT r.orgtypeid, r.name FROM rolepermission rp
JOIN role r ON r.id = rp.roleid
JOIN permission p ON p.id = rp.permid
WHERE p.path = '/the/exact/path';

-- 3. What roles does the user have?
SELECT r.orgtypeid, r.name FROM userrole ur
JOIN role r ON r.id = ur.roleid
WHERE ur.userid = '<userid>';

-- 4. Cross-reference: user's role keys (e.g. reseller.admin) must appear in step 2.
--    Note: get_userroles() auto-expands to *.name and orgtypeid.* wildcards,
--    so a user with reseller.admin also matches *.admin and reseller.*.

RBAC cache: UserPermissions.rp_caches has 600s TTL. After DB changes, either restart Sage or wait for cache expiry.

18. Allow Same Phone Number for Multiple User Registrations

sms_register.dspy blocks registration if phone exists. Remove the sor.R('users', {'mobile': mobile}) check. code_login.dspy and phone_login.dspy already handle multi-user-per-phone via selection list UI.

19. get_search_roleid.dspy — Return ALL Roles (Not Org-Type Filtered)

Problem: get_search_roleid.dspy filtered by current user's org_type. The code uitype in list view resolves stored roleid values — if a role belongs to a different org_type, it's missing from filtered results and displays as raw ID.

Fix: Remove org_type filter, return all non-system roles unconditionally:

SELECT id, CONCAT(orgtypeid, '.', name) as name FROM role
WHERE id NOT IN ('anonymous', 'any', 'logined')
AND orgtypeid != '*' AND name != '*'
AND orgtypeid IS NOT NULL AND name IS NOT NULL
ORDER BY orgtypeid, name

permission.id VARCHAR(32) Truncation

Problem: permission.id is generated as perm_ + path with / replaced by _. Long paths like /knowledge_bases_list/upload.dspy → perm__knowledge_bases_list_upload.dspy (39 chars) exceed the VARCHAR(32) column limit.

Consequence: MySQL silently truncates the ID to 32 chars (perm__knowledge_bases_list_uploa). rolepermission.permid is also truncated. Since both are truncated identically, the JOIN still works — but the warning Data truncated for column 'id' appears in logs. Two different long paths could theoretically collide but in practice this is rare.

Fix: Not strictly necessary for correctness (truncation is consistent), but can be avoided by using shorter path names or manually crafting shorter permission IDs.

Verification Commands

# Check perm_config.py has no /main prefix
grep '/main/' perm_config.py  # Should return nothing

# Test login + permission
curl -c /tmp/cookie.txt -X POST http://localhost:8080/rbac/user/up_login.dspy \
  -d 'username=super&password=Kyy@123456'
curl -b /tmp/cookie.txt http://localhost:8080/customer_management/customer_list.ui

Admin User Creation

  • Role-based tier analysis: See references/harnessed-permission-analysis.md for a complete walkthrough of how to analyze module business nature and classify paths into Public/Logined/Admin/Superuser tiers.
  • Sage deployment patterns: See references/sage-deployment-patterns.md for module installation, integration, four-tier permission model, role definitions, and permission init script template.
  • Module-level RBAC script template: See references/per-module-load-path.md for the scripts/load_path.py Python template — direct DB operations, idempotent path registration, role assignment guide.
  • Admin user creation: See references/admin-user-creation.md for creating superuser accounts with password_encode() bcrypt hashing, org creation, and role assignment.
  • User self-registration flow: See references/user-registration-flow.md for the register_user function, three registration entry points, role customization during registration, and profile editing form patterns.
  • Transfer payment email scanning: See references/transfer-payment-patterns.md for POP3 email parsing, regex patterns, DSPY→Python delegation, and common pitfalls.
  • ahserver auth middleware: See references/ahserver-auth-middleware.md for the full middleware architecture, how all requests (including static files) pass through RBAC, common failure modes (500 on static resources, DB connection failures, cache miss slowdowns), and cache details.
  • WebSocket debugging: See references/websocket-debugging.md for WSS architecture, three path contexts, frontend/ahserver pitfalls, and deployment checklist.
  • RBAC init script: See references/rbac-init-script.md for a complete standalone Python script that creates any/logined roles and registers permissions — useful for independent apps without Sage's set_role_perm.py.