pipeline_core/skills_library/all/module-development-spec/SKILL.md

name	version	description	trigger_conditions
module-development-spec	1.0.0	Standardized development workflow for creating modular components following the specified directory structure, frontend/backend conventions, and database integration patterns.	User requests to create a new module named "mymodule" or similar Task involves implementing a component with bricks-framework frontend and sqlor database backend Development follows the documented directory structure with wwwroot, models, json, and init directories Task involves creating read-only/dashboard modules that display data without CRUD (no models/json needed)

Work logs / delivery archive

For production module/application development, update a dated work log before finishing the session. Use the actual current date from the system (do not assume the date from earlier messages) and keep late-night follow-up fixes under their real date or explicitly label them as next-day follow-ups. If no log file exists, create one under an appropriate docs/archive path such as docs/work-log-YYYY-MM-DD.md.

A useful log entry should include:

• Scope/background and repository/module name.
• Timeline or commit list for the date.
• Key technical decisions and pitfalls discovered.
• Verification performed and any environment-limited verification that could not run.
• Current branch/commit state.

Do not leave a task as "done" without this archive when the user expects deliverables to be retained for later inspection.

This skill defines the complete workflow for developing standardized modules that integrate with the ahserver ecosystem using bricks-framework for frontend and sqlor for database operations.

Module Philosophy — Host-Agnostic

Modules are NOT tied to any specific application. A module is a self-contained unit that any host can load:

• pipeline-app loads it → it's pipeline-app's module
• sage loads it → it's sage's module
• Any future application loads it → it's that application's module

Core principle: A module only depends on:

1. Foundation packages (sqlor, ahserver for ServerEnv, appPublic for utilities)
2. Its own data tables (or shared tables like sage's appcodes)
3. Other modules it explicitly imports

A module does NOT depend on:

• Any specific host application's entry point
• Host-specific configuration files
• Host-specific wwwroot paths

The load_{module}() function is the ONLY integration point. It registers functions to ServerEnv. The host decides how to wire it in.

Interaction-layer modules (like pipeline-task) have NO data tables — they are pure thin wrappers that call other modules' functions via ServerEnv. They provide .dspy + .ui files for user interaction only.

Directory Structure

mymodule/                 # Main module directory (replace mymodule with actual name)
├── mymodule/             # Python package directory
│   ├── __init__.py       # Required Python package file
│   ├── init.py           # Module initialization script
│   └── *.py              # Additional source files
├── wwwroot/              # Frontend scripts and resources (.ui, .dspy files)
├── models/               # Database table definitions as JSON files
├── json/                 # CRUD operation definitions as JSON files  
├── init/                 # Module initialization data
│   └── data.json         # Initial data in specified format
├── scripts/              # Supporting scripts
│   └── load_path.py      # RBAC permission registration script (see RBAC section below)
├── skill/                # MANDATORY — AI agent reference documentation
│   └── SKILL.md          # Agent-facing module spec: data model, endpoints, pitfalls
├── pyproject.toml        # Python packaging configuration
└── README.md             # Module documentation

Core Implementation Requirements

1. Module Initialization (init.py)

• Primary purpose: Register all module functions with ServerEnv instance so they can be called directly from .ui and .dspy files
• NOT for route registration: wwwroot files are automatically routed via /{module_name}/filename.ext
• ServerEnv configuration is REQUIRED for all modules: Every module must register its functions with ServerEnv in the load_{modulename}() function
• CRITICAL: Export functions in init.py: All async functions defined in init.py must be explicitly imported in the package's __init__.py file, otherwise Sage framework cannot find them and dspy calls will fail with NameError: name 'xxx' is not defined

Pitfall: Triple-place function registration — adding or removing functions requires updating THREE files

When adding or removing a module function, THREE files must be updated in sync:

1. mymodule/mymodule.py (or wherever the function is defined) — the function implementation
2. mymodule/__init__.py — the import/export line
3. mymodule/init.py — the env.xxx = xxx registration in load_{module}()

Forgetting any one causes runtime errors:

• Missing #2 → ImportError or AttributeError when init.py tries to import
• Missing #3 → NameError: name 'xxx' is not defined in .dspy/.ui templates
• Removing from #1 but not #2/#3 → ImportError at module load time

Cleanup checklist when removing a function:

grep -rn 'function_name' mymodule/ --include='*.py'
# Must appear in exactly: definition, __init__.py import, init.py env registration
# Remove from ALL three, then commit.

Example __init__.py (required):

# mymodule/__init__.py
from .init import (
    create_user,
    get_user_list,
    update_user_profile,
    # ... export all public API functions
)

• Keep init.py focused on ServerEnv registration and minimal initialization
• Include a load_{modulename}() function that registers all public functions with ServerEnv
• Functions registered with ServerEnv become directly callable from .ui templates and .dspy scripts

Example structure:

from ahserver.serverenv import ServerEnv

# Module metadata and constants  
MODULE_NAME = "mymodule"
MODULE_VERSION = "1.0.0"

# Helper functions for web scripts
def get_data():
    """Get data - will be registered with ServerEnv"""
    pass

def save_data(data):
    """Save data - will be registered with ServerEnv"""  
    pass

def load_mymodule():
    """Register all functions with ServerEnv so they can be called from .ui/.dspy files"""
    env = ServerEnv()
    env.get_data = get_data
    env.save_data = save_data
    return True

Pitfall: CRUD dspy wrappers use plural table names — register both singular and plural

The xls2ui tool generates CRUD wrapper .dspy files (e.g., add_suppliers.dspy) that reference functions with plural names matching the table name: create_suppliers, update_suppliers, delete_suppliers. But init.py typically defines functions with singular names (e.g., create_supplier). If only the singular name is registered, the CRUD UI will get NoneType errors.

Fix: Register both forms in load_{module}():

env.create_supplier = create_supplier
env.create_suppliers = create_supplier  # xls2ui uses plural
env.update_supplier = update_supplier
env.update_suppliers = update_supplier
env.delete_supplier = delete_supplier
env.delete_suppliers = delete_supplier

Symptom: CRUD "add" button returns 500 with function not found — the wrapper dspy calls create_suppliers but only create_supplier was registered.

Pitfall: Subagent Delegation for Module Creation — ALWAYS Validate After

When using delegate_task to create modules in parallel, subagents consistently produce errors even when given correct spec instructions (observed 2/3 failure rate):

1. File misplacement: init.py and __init__.py placed at module root (mymodule/init.py) instead of package dir (mymodule/mymodule/init.py)
2. Wrong model JSON format: Uses {"table":"...", "fields":{...}} instead of {"summary":[...], "fields":[...], "indexes":[...], "codes":[...]}
3. Wrong CRUD JSON format: Uses {"table":"...", "list":{...}} instead of {"tblname":"...", "params":{"browserfields":{...}, "editable":{...}}}
4. Hallucinated sqlor APIs: Invents sqlor.save/list/one/delete/insert/query — these methods DO NOT EXIST. Only sor.C/U/D/R/I/sqlExe are valid.

Mandatory post-delegation validation (see references/bulk-module-creation-pattern.md "Subagent Validation Checklist"):

• Move misplaced init.py/init.py to package dir
• Validate all model JSONs have summary key with array primary
• Validate all CRUD JSONs have tblname and params.editable
• Grep for hallucinated sqlor APIs and fix to sor.C/U/D pattern
• Run dspy audit (no imports, no print, no uuid)
• Run py_compile on all .py files

Pitfall: debug() in dspy does NOT include filename — always prefix manually

debug() calls in .dspy files log as [sage][debug][<string>:N] — the framework injects dspy code into a <string> context, so the file path is never shown. When tracing production issues across multiple dspy files, all debug output looks identical.

Fix: Always prefix debug calls with the file name:

# WRONG — no way to tell which dspy this came from
debug(f'{params_kw=}')

# CORRECT — immediately identifiable in logs
debug(f'product_category_create.dspy: START params_kw={dict(params_kw)}')
debug(f'get_product_category.dspy: ns keys={list(ns.keys())}')

This is essential when debugging multi-step flows (e.g. add → refresh) where several dspy files execute in sequence. Each debug line becomes self-identifying.

Pitfall: DSPY API handlers must forward ALL client params — never hardcode dispatch fields

When a .dspy API endpoint acts as a task dispatcher (submitting to a longtasks worker or downstream service), it MUST forward the client's parameters. Hardcoding dispatch fields like task_type silently ignores client intent:

# WRONG — hardcodes task_type, ignores client's task_type
payload = {'task_type': 'separate', 'task_id': task_id, 'audio_path': audio_path}
await longtasks.submit_task(payload)

# CORRECT — reads task_type from request, forwards all params
task_type = params_kw.get('task_type', 'separate')
payload = {'task_type': task_type, 'task_id': task_id, 'audio_path': audio_path}
if params_kw.get('output_dir'):
    payload['output_dir'] = params_kw['output_dir']
await longtasks.submit_task(payload)

Symptom: Client sends task_type: "separate_full" but worker log shows task_type: "separate". Worker always runs the default mode regardless of what client requested. Takes hours to trace because the worker code is correct — the DSPY is silently dropping the parameter.

Audit: grep API DSPY files for hardcoded 'task_type', 'mode', or similar dispatch strings inside payload construction.

Pitfall: py_compile is INVALID for .dspy files

.dspy files are injected into an async function context at runtime — return at top level is correct. Running py_compile on .dspy files produces false positives ('return' outside function, 'await' outside function). Only use py_compile for .py files (like init.py). For .dspy validation, use the dspy audit script (grep-based checks).

CRITICAL: dspy MUST use explicit return — ahserver wraps code in async def. The dspy handler prepends async def myfunc(request, **ns): and awaits the function. Setting result as a bare expression without return causes the function to return None. See references/dspy-execution-and-module-structure.md for the full execution model, pre-loaded globals, Tree widget data format, and module symlink patterns.

Pitfall: Sage startup requires environment variables — cannot test locally without them

Sage's app/sage.py imports modules that read env vars at import time (e.g., ALIPAY_PUB for payment keys). Starting Sage locally without these vars causes FileNotFoundError. Use syntax checks (py_compile for .py files) and dspy audit scripts instead of full startup for local validation.

Pitfall: Raw SQL column names — ALWAYS verify against model.json, never guess

When writing raw SQL (sor.sqlExe) or ORM calls that reference table columns, ALWAYS verify the column names against the actual model definition (models/{table}.json), not from memory or similar tables. Guessing column names (e.g., permcode instead of path, permname instead of name) causes hard-to-debug runtime errors that require multiple fix attempts.

Check before writing SQL:

python3 -c "import json; cols=[f['name'] for f in json.load(open('models/table.json'))['fields']]; print(cols)"

Common wrong guesses → correct names in Sage:

Table	Wrong guess	Correct (from model)
permission	permcode, permname	path, name
users	created_date	created_at
organization	org_name	orgname

Pitfall: {{id}} in uapi response templates resolves to Python built-in id()

In Sage uapi response templates, Jinja2 resolves {{id}} as Python's id() built-in function, not the upstream API's id field. This produces <built-in function id> in the response instead of the actual task/request ID.

Fix: Never use {{id}} in uapi response templates. Always map to a different field name:

# WRONG — {{id}} resolves to Python id() function
'{"taskid":"{{id}}","status":"{{status}}"}'

# CORRECT — map via request context or use a different variable
'{"taskid":"{{taskid}}","status":"{{status}}"}'

Affected: Any uapi where the upstream response has a field called id. Sage's Jinja2 context silently shadows it with the Python built-in.

Pitfall: uapiio input_fields MUST be a JSON array for bricks.js compatibility

The bricks.js LlmIO widget calls this.input_fields.forEach(...). If input_fields is a JSON object {"field": {...}} instead of a JSON array [{"name":"field", ...}], the browser throws:

TypeError: this.input_fields.forEach is not a function

Fix: Always format uapiio input_fields as an array of objects:

// WRONG — object format, bricks.js can't iterate
{"prompt":{"type":"string","required":true}}

// CORRECT — array format with name/label/uitype
[{"name":"prompt","label":"用户输入","uitype":"text","required":true}]

Reference: Use existing KTV uapiio records (e.g., ktv_asr_transcribe_io) as the canonical format reference.

Pitfall: json.dumps() in uapi data templates causes data=None

When a uapi data template uses {{json.dumps(prompt, ensure_ascii=False)}}, the Jinja2 rendering can fail silently, resulting in the HTTP request body being None. This logs as body=None in the uapi debug output.

Fix: Use plain string interpolation "{{prompt}}" instead of json.dumps(). The model name should be hardcoded to the upstream service ID (e.g., "model":"hy-image-v3.0"), not passed through {{model}}.

# WRONG — json.dumps can render to None
'{"model":"{{model}}","prompt":{{json.dumps(prompt,ensure_ascii=False)}}}'

# CORRECT — plain string, hardcoded service model ID
'{"model":"hy-image-v3.0","prompt":"{{prompt}}"}'

Model ID sourcing: The model ID sent to the upstream must be the provider's actual service ID. Discover actual IDs by querying the provider's /v3/models endpoint (ARK) or /v1/models endpoint (tokenhub), then hardcode them in the uapi data template.

Pitfall: return inside async with → silent NoneType in .dspy files

The ahserver framework wraps .dspy code in async def myfunc(request, **ns): and awaits it. Returning from inside an async with db.sqlorContext(...) as sor: block causes the function to return None, producing return data type error, <class 'NoneType'>.

Fix: Collect all results inside the async with block, then return AFTER the block exits:

# WRONG — return inside async with → NoneType
async with db.sqlorContext(dbname) as sor:
    recs = await sor.R('table')
    return json.dumps({'total': len(recs), 'rows': rows})

# CORRECT — collect inside, return outside
rows = []
async with db.sqlorContext(dbname) as sor:
    recs = await sor.R('table')
    for r in (recs or []):
        rows.append({'id': r.id, 'name': r.name})
return {'total': len(rows), 'rows': rows}

Symptom: Exception: /path/file.dspy return data type error, <class 'NoneType'> with no other traceback. Minimal test: return {'ok': True} inside async with also fails.

Pitfall: sor.C() silently drops records when created_at is missing

sqlor's sor.C() does NOT auto-set timestamp fields. If the model defines created_at TIMESTAMP NOT NULL, sor.C may succeed silently but the record never reaches the database. Always include 'created_at': curDateString():

ns = dict(params_kw)
ns['id'] = params_kw.get('id', getID())
ns['created_at'] = curDateString()   # REQUIRED — sor.C does not auto-set
ns['org_id'] = (await get_userorgid()) or '0'
async with db.sqlorContext(dbname) as sor:
    await sor.C('table_name', ns)

Symptom: dspy returns {"success": true} but SELECT returns zero rows.

Pitfall: Bash heredoc \$ escapes when writing dspy files via terminal

When using cat > file.dspy << 'EOF' in bash, backslash sequences like \${pid}\$ intended as sqlor placeholders get written literally as \${pid}\$ instead of ${pid}$. This causes sqlor to fail silently because the placeholder pattern is unrecognized.

Fix: Never escape $ in heredocs for dspy files. Write ${pid}$ as-is:

# WRONG — heredoc writes literal backslash
cat > file.dspy << 'EOF'
sql = 'WHERE id=\${pid}\$'
EOF

# CORRECT — no escaping needed in single-quoted heredoc
cat > file.dspy << 'EOF'
sql = 'WHERE id=${pid}$'
EOF

Verification: grep 'project_id' file.dspy | cat -A — look for \$ (shows as \$) which means the backslash was written literally. Correct output shows ${pid}$ with no backslash.

Pitfall: Do NOT name module functions get_module_dbname

In init.py, do NOT define a function named get_module_dbname() — this name is already provided as a global by ServerEnv and is used by .dspy scripts. Registering it via env.get_module_dbname = ... will overwrite the global. If you need a module-specific dbname helper, name it with a private prefix like _get_dbname().

Pitfall: Do NOT name module functions get_module_dbname

Key Points:

• All functions that need to be accessible from .ui/.dspy files must be registered with ServerEnv
• The load_{modulename}() function is called by Sage during module loading
• After registration, functions can be called directly in .ui templates: {% set data = get_data() %}
• This pattern is used by all standard modules (rbac, appbase, etc.)

2. Frontend Development

• Mandatory: Use bricks-framework for all frontend interfaces
• Store all .ui files as pure JSON format (NOT HTML/CSS) in wwwroot/
• See references/bricks-ui-pitfalls.md for id placement (widget level, not options), Button click event, Popup/Form patterns, and script actiontype Jinja2 limitations.
• Automatic routing: Files in wwwroot/ are automatically accessible via /{module_name}/filename.ext
• No manual route registration needed: The framework automatically serves .ui and .dspy files
• .ui files contain component definitions with widgettype, options, subwidgets, and binds properties
• Store CSS files in wwwroot/ for styling extensions
• Store JavaScript files in wwwroot/ for custom functionality and registerFunction extensions
• ahserver auto-serves .css and .js files: files placed in wwwroot/ are automatically discovered and injected by ahserver into the HTML response — DO NOT manually add <link> or <script> tags in .ui files to reference them. This is a common mistake that causes unnecessary 403 errors when RBAC is enabled.
• WARNING: Even though ahserver auto-serves .css/.js files, they are STILL subject to RBAC checks. Every .css and .js file must be registered in load_path.py. Files needed before authentication (e.g., theme CSS) should use any role.
• Store .dspy files (controlled Python scripts) in wwwroot/ for backend logic
• The build.sh script will link module wwwroot files to the main application wwwroot directory

2.0 Module Entry Point (index.ui) — MANDATORY

• Every business module MUST have wwwroot/index.ui as its user-facing entry point
• Exception: reference/foundation modules (rbac, appbase, accounting, apppublic, sqlor, ahserver) are exempt
• index.ui should integrate ALL the module's .ui and .dspy files into a single navigation page
• Use ResponsableBox with clickable card widgets (VBox + binds click → urlwidget) to link to each feature page
• Use {{entire_url('filename.ui')}} or {{entire_url('api/xxx.dspy')}} for ALL url values — this converts server-relative paths to the correct runtime URL prefix
• Default content area: a VBox with id: "app.<module>_content" for urlwidget targets to load pages into

Example index.ui pattern:

{
    "widgettype": "VBox",
    "options": {"width": "100%", "height": "100%", "padding": "20px"},
    "subwidgets": [
        {"widgettype": "Text", "options": {"label": "模块名称", "fontSize": "24px"}},
        {"widgettype": "ResponsableBox", "options": {"gap": "16px", "minWidth": "250px"},
         "subwidgets": [
            {"widgettype": "VBox",
             "options": {"backgroundColor": "#FFFFFF", "padding": "20px", "cursor": "pointer"},
             "binds": [{
                 "wid": "self", "event": "click", "actiontype": "urlwidget",
                 "target": "app.module_content",
                 "options": {"url": "{{entire_url('feature_page.ui')}}"},
                 "mode": "replace"
             }],
             "subwidgets": [{"widgettype": "Text", "options": {"label": "功能名称"}}]}
         ]},
        {"widgettype": "VBox", "id": "module_content",
         "options": {"width": "100%", "flex": "1", "marginTop": "20px"}}
    ]
}

2.1 URL Path Rules — MANDATORY

• ALL url fields in .ui files MUST use {{entire_url(url)}} format
• This applies to: Menu items, Button url, Form url, Iframe url, urlwidget targets, and any other url property
• {{entire_url()}} converts server-relative paths (e.g., feature.ui, /module/page.ui) to the correct runtime URL including any prefix configured in config.json
• WRONG: "url": "/module/page.ui" or "url": "feature.ui"
• CORRECT: "url": "{{entire_url('/module/page.ui')}}" or "url": "{{entire_url('feature.ui')}}"
• Do NOT use {{entire_url()}} for: data: URIs, # anchors, http(s):// URLs, or variable references ${var}

2.2 JSON CRUD File URL Rules — MANDATORY

• json/ 目录下的 CRUD 定义文件（如 json/handover_list.json）使用 entire_url() 引用同模块的其他 CRUD 别名时，必须使用 ../ 相对路径
• 原因：JSON 文件位于 json/ 子目录中，而 .dspy CRUD 文件和生成的 .ui 文件位于模块根目录（即 wwwroot/），需要跳出 json/ 目录
• 适用于：subtables[].url、browserfields 中的 dataurl 等所有 entire_url() 引用
• WRONG: "url": "{{entire_url('handover_items_list')}}"\n- CORRECT: "url": "{{entire_url('../handover_items_list')}}"
• 引用其他模块的别名（如 appcodes_list）不需要 ../，因为通过模块间路由解析

2.2a CRUD JSON new_data_url — Do NOT Pass editexclouded Fields as Query Params

Pitfall: When new_data_url passes a field as a query param (?supplier_id=...) that is also in editexclouded, the Form POST body and URL query params get merged into params_kw as a list (e.g., supplier_id: ['val', 'val']). sor.C receives a list instead of a string, MySQL interprets it as a subquery, and throws Operand should contain 1 column(s).

Fix: remove the query param from new_data_url. If the field is excluded from the form (editexclouded), don't pass it in the URL. If it needs to be pre-set for subtable context, add it as a hidden field in the form instead.

// WRONG
"new_data_url": "{{entire_url('../api/create.dspy')}}?supplier_id={{params_kw.get('supplier_id','')}}"

// CORRECT
"new_data_url": "{{entire_url('../api/create.dspy')}}"

• 适用于：subtables[].url、browserfields 中的 dataurl 等所有 entire_url() 引用
• WRONG: "url": "{{entire_url('handover_items_list')}}"
• CORRECT: "url": "{{entire_url('../handover_items_list')}}"
• 引用其他模块的别名（如 appcodes_list）不需要 ../，因为通过模块间路由解析

2.3 JSON CRUD File editable Section — MANDATORY

• 所有用于 DataViewer/Tabular 列表展示的 JSON 配置文件必须包含 editable 段，否则"新增/编辑/删除"按钮无法提交数据
• editable 段定义表单提交的三个目标 URL，指向 wwwroot/api/ 下的 .dspy 脚本
• 重要：wwwroot 是 Web 根目录，在 URL 层不可见，路径中不能出现 wwwroot/
• 模板中的 CRUD 别名在 json/ 目录下，.dspy 文件实际在 wwwroot/api/，但 URL 路径用 ../api/（跳过 wwwroot）
• WRONG: JSON 中没有 new_data_url/update_data_url/delete_data_url，或者路径包含 wwwroot，或者这些字段嵌套在 editable 对象中
• CORRECT — new_data_url/update_data_url/delete_data_url 必须放在 params 顶层，不能嵌套在 editable 对象内：

{
    "tblname": "tablename",
    "alias": "alias_name",
    "title": "模块名称",
    "params": {
        "browserfields": { ... },
        "new_data_url": "{{entire_url('/module/api/tablename_create.dspy')}}",
        "update_data_url": "{{entire_url('/module/api/tablename_update.dspy')}}",
        "delete_data_url": "{{entire_url('/module/api/tablename_delete.dspy')}}"
    }
}

Pitfall: editable nesting causes new_data_url to be silently ignored. The xls2ddl template reads new_data_url from params top level ({% if new_data_url %}), NOT from params.editable.new_data_url. Nesting these fields inside an editable object causes xls2ui to generate the default add_{tablename}.dspy instead — the CRUD falls back to framework default insert, and any custom create logic is silently bypassed.

Real case (2026-07-13): discount module had "editable": {"new_data_url": "...", ...} nested. xls2ui generated add_discount_marketing.dspy instead of using marketing_create.dspy. created_by field was never set by custom handler. Fixed by moving to params top level and using /module/api/ absolute paths.

• _edit.json 文件（表单编辑页配置）同样需要 editable 段
• 三个 .dspy 文件必须存在于 wwwroot/api/ 目录：xxx_create.dspy、xxx_update.dspy、xxx_delete.dspy
• 如果缺少 editable 段，DataViewer 的 build_add_form() / build_update_form() 会因为 this.editable.new_data_url 为 undefined 而提交失败
• 完整 JSON 配置模板见 references/json-config-pattern.md
• 自定义工具栏按钮 + PopupWindow bind 模式见 references/crud-json-toolbar-bind-pattern.md

2.3b bricks Toolbar Bind — Only 5 Valid Actiontypes

Pitfall: bricks.js bind 的 actiontype 只有 5 种有效值：urlwidget、script、url、datawidget、event。dspy 和 function 不是有效 actiontype。工具栏调用 dspy 端点的正确模式：

{
    "actiontype": "urlwidget",
    "target": "PopupWindow",
    "popup_options": {"title": "标题", "height": "200px", "width": "400px"},
    "params_mapping": {"mapping": {"id": "target_param"}, "need_other": false},
    "options": {"method": "POST", "url": "{{entire_url('../api/xxx.dspy')}}"}
}

dspy 返回值被 bricks 渲染在 PopupWindow 中。如果 dspy 返回 DataViewer widget 嵌入 CRUD 页，url 必须带 _webbricks_=1 参数。

禁止：script actiontype 中使用 fetch() / bricks_fetch() — bricks 原则明确禁止 script 中的 fetch/setInterval。

2.3c Module-Internal entire_url() Must Use Absolute /module/ Paths

Pitfall: 模块内 .ui 文件（index.ui、menu.ui）中 entire_url('supply_contracts_list') 可能解析为 /supply_contracts_list（缺模块前缀 /supplychain/），RBAC 检查用缺前缀路径 → 403。

Fix: 所有模块内部的 URL 使用绝对路径：

{# WRONG: 缺模块前缀 → 403 #}
{{entire_url('supply_contracts_list')}}

{# CORRECT #}
{{entire_url('/supplychain/supply_contracts_list')}}

检查方法：

grep "entire_url(" wwwroot/index.ui wwwroot/menu.ui | grep -v '/supplychain/' && echo "MISSING PREFIX"

2.3d dspy Returned DataViewer URL Must Be Absolute

Pitfall: dspy 返回的 DataViewer widget 中相对路径 ../distribution_agreements_list 在 PopupWindow 上下文中解析错误，缺模块前缀。

Fix: dspy 中用绝对路径（entire_url() 是 Jinja2 函数，dspy 中不可用）：

return {
    'widgettype': 'DataViewer',
    'options': {
        'url': f'/supplychain/supply_contracts_list?supplier_id={supplier_id}&_webbricks_=1'
    }
}

2.4 Menu Widget — Do NOT use binds

• Menu控件已经内部处理了click事件，不需要额外添加 binds 绑定
• Menu items 的 url 字段会自动处理导航，只需设置 url: "{{entire_url('page.ui')}}" 即可
• WRONG: 在 Menu 上添加 binds 处理 itemclick
• CORRECT: 在 Menu items 中设置 url，Menu 自动处理点击

2.1 DSPY File Implementation Guidelines

• Purpose: .dspy files provide backend API endpoints that can be called from .ui files
• HTTP Method Handling: ahserver supports GET and POST; the framework distinguishes automatically
• Parameter Extraction: ALL parameters (query string AND POST body) are automatically parsed by ahserver into params_kw — a dictionary-like object. Access via params_kw.get('key') or params_kw.key. Nested JSON objects/arrays in POST bodies are preserved as dict/list structures in params_kw
• POST Data Reading: Do NOT use os.environ, os.read(), or sys.stdin. ahserver reads and parses POST data automatically
• Error Handling: Always return consistent JSON response structure with status, data, and optional message/total fields
• Security: Validate all input parameters and implement proper error handling
• Integration: Call functions from your module's init.py (registered with ServerEnv via load_{modulename}()) to perform business logic
• Debug output: Use debug_params('name', dict_obj) for compact logging instead of debug(f'{dict_obj=}') which dumps entire objects. See references/web-app-and-dspy-spec.md for full debug output best practices.
• Extending .dspy globals: To make a new function available in all .dspy files, inject it via self.y_env in ahserver/processorResource.py. See references/web-app-and-dspy-spec.md section "Injecting New Functions into .dspy Execution Environment".

CRITICAL: ServerEnv() is NOT needed in .dspy scripts

• DO NOT write env = ServerEnv() or env.get_module_dbname('xxx') in .dspy files
• All ServerEnv-registered functions are directly available via globals(): get_module_dbname(), get_user(), get_userorgid(), DBPools(), password_encode(), getID(), module-specific functions like create_user_apikey, etc.
• WRONG: env = ServerEnv(); dbname = env.get_module_dbname('dapi'); create_apikey_func = getattr(env, 'create_user_apikey', None)
• CORRECT: dbname = get_module_dbname('dapi'); create_apikey_func = create_user_apikey
• CORRECT for org_id: org_id = (await get_userorgid()) or '0'
• Only use ServerEnv() in Python module files (init.py, *.py), never in .dspy scripts

Pitfall: ServerEnv() is a process-level singleton. Every call to ServerEnv() within a process returns the SAME instance. So env = ServerEnv() in load_*() and env = ServerEnv() deep in accounting.py share state. env.get_module_dbname = ... set in backend_accounting.py is visible everywhere. Do NOT assume each call creates a new instance.

Pitfall: ServerEnv() ≠ request._run_ns — NEVER use ServerEnv() to get per-request user/session context. Per-request functions (get_user(), get_userid(), get_userorgid()) are registered on request._run_ns (which is self.y_env, a plain DictObject set up in processorResource.py:347-349), NOT on the ServerEnv() singleton. Using env = ServerEnv(); await env.get_user() silently returns None.

Wrong (user fields silently become NULL):

async def create_record(request, params_kw):
    env = ServerEnv()                          # WRONG — singleton, no per-request user
    dbname = env.get_module_dbname('module')   # works (set at startup)
    user_id = await env.get_user()             # SILENTLY RETURNS None
    org_id = await env.get_userorgid()         # SILENTLY RETURNS None
    await sor.C('table', {'created_by': user_id})  # created_by = None! Bug.

Correct:

async def create_record(request, params_kw):
    env = request._run_ns                      # CORRECT — per-request namespace
    dbname = env.get_module_dbname('module')   # also works (via RF mechanism)
    user_id = await env.get_user()             # Returns actual user ID
    org_id = await env.get_userorgid()         # Returns actual org ID

When IS ServerEnv() correct?

• In load_{module}() — runs once at startup, registers global functions
• For startup-time globals when no request is available

Detection command:

grep -n 'env = ServerEnv()' module/init.py | while read line; do
  lineno=$(echo "$line" | cut -d: -f1)
  sed -n "$((lineno+1)),$((lineno+6))p" module/init.py | grep -q 'get_user\|get_userorgid\|get_userid' \
    && echo "LINE $lineno: BUG — ServerEnv() has no per-request user context"
done

Real case (2026-07-13): discount module create_marketing used env = ServerEnv() → created_by always NULL. 7 functions affected. Same root cause as generate_promo_code which correctly used request._run_ns.

Pitfall: Module pkgs go under SAGE, not cross-applications. A module's wwwroot symlink must point to ../pkgs/<module>/wwwroot under the HOST application's directory. Do NOT place modules under another app's pkgs (e.g., /pipeline-app/pkgs/tenant/) — this breaks the module's independence. Each application clones installs modules into its own pkgs/ directory.

Example: tenant module should be at /d/apitest/sage/pkgs/tenant/ with symlink sage/wwwroot/tenant -> ../pkgs/tenant/wwwroot. NOT at /d/apitest/pipeline-app/pkgs/tenant/.

Pitfall: sqlor method argument counts differ — sor.I is special

Method	Args	Pattern
sor.C	2	sor.C('table', ns)
sor.R	2	sor.R('table', ns)
sor.U	2	sor.U('table', ns)
sor.D	2	sor.D('table', ns)
sor.I	1	sor.I(ns)

sor.I(data) only takes the data dict — no table name. Passing sor.I('table', data) causes TypeError: I() takes 2 positional arguments but 3 were given. The table name is resolved from the sqlor context.

Read-Only / Dashboard Modules

Not all modules need database tables or CRUD operations. For read-only modules (dashboards, statistics, reporting), models/ and json/ directories can be omitted entirely.

Preferred pattern: load_XX.py exposes async data functions via ServerEnv → individual .ui files use Jinja2 templates ({{get_data(request)}}) → RefreshWidget wraps each card for auto-refresh. No .dspy files or JS polling needed.

See references/read-only-module-pattern.md for:

• Jinja2 .ui + RefreshWidget + load_XX architecture (preferred)
• Legacy .dspy + JS polling pattern (deprecated, still documented)
• ECharts ChartBar/ChartLine integration with inline data or data_url
• Cross-table queries using the shared 'sage' database
• Concurrent user detection pattern

Database Integration

• Mandatory: Use sqlor-database-module for all relational database operations
• Store table definitions as {tablename}.json in models/ directory
• Create CRUD definition files as {tablename_or_alias}.json in json/ directory
• Follow sqlor database operation specifications
• Bulk xlsx→json conversion: use scripts/xlsx2json_models.py — converts all .xlsx model files across Sage modules to JSON format in one pass

4. Initialization Data

• Store in init/data.json or init/data.yaml. Two formats are supported:

Format A — Direct table seeding (arbitrary tables, JSON):

{
    "table1": [
        {"field1": "value1", ...},
        {"field1": "valuen", ...}
    ]
}

Format B — Appcodes registration (code value groups for appcodes_kv, JSON):

{
    "appcodes": [
        {
            "parentid": "sc_relation_type",
            "parentname": "供销关系合作类型",
            "items": [
                {"k": "distribution", "v": "分销"},
                {"k": "agency", "v": "代理"}
            ]
        }
    ]
}

Format C — Appcodes registration (YAML) — used by accounting and some other modules:

appcodes:
- id: credit_status
  name: 信用额度状态
  hierarchy_flg：0

appcodes_kv:
- id: credit_status_active
  parentid: credit_status
  k: active
  v: 生效
- id: credit_status_inactive
  parentid: credit_status
  k: inactive
  v: 停用

YAML format differences from JSON Format B:

• appcodes entries use id (primary key), name, hierarchy_flg (note: some files use fullwidth colon ： for hierarchy_flg — this is a known pre-existing quirk)
• appcodes_kv entries use id, parentid, k, v — each entry is a flat record (not nested under items)
• The parentid in appcodes_kv must match an id from the appcodes section
• When adding new code groups, add BOTH the appcodes parent record AND all appcodes_kv child records

Pitfall: YAML lint warnings about hierarchy_flg：0 — Some existing data.yaml files use a fullwidth colon (：) instead of ASCII colon (:) for hierarchy_flg. This is a pre-existing issue in the file, not something introduced by new edits. Do not "fix" it unless the user asks — it may break the loader that expects the original format.

5. Encoding Management

• Store encodings in appbase module's appcodes and appcodes_kv tables
• appcodes table structure:
  • id: str(32), primary key (can be field name)
  • name: encoding name
  • hierarchy_flg: str(1), '0'=single-level, '1'=multi-level
• appcodes_kv table structure:
  • id: str(32), primary key
  • parentid: str(32), references parent encoding
  • k: str(32), encoding value
  • v: str(255), display text

Pitfall: init/data.json parentid too long → appcodes_kv.id exceeds VARCHAR(32)

The init data loader generates appcodes_kv.id as {parentid}_{k}. Both id and parentid columns are VARCHAR(32). If parentid is too long, the derived ID overflows. Example: supplier_settlement_cycle (27 chars) + _ + quarterly (9 chars) = 37 chars → DataError: Data too long for column 'id'.

Rule: len(parentid) + 1 + len(longest_k) ≤ 32. Parentid must be ≤ 22 chars when k values are typical code words. Keep parentids short (≤ 22 chars). When already referenced in models/*.json codes cond, update both init/data.json and all models/*.json files referencing the old parentid.

5b. Codes Referencing Future Modules

When a table field references another module that does not yet exist (e.g., productid → products table from a future product module), still add the codes entry in the model JSON:

"codes": [
    {
        "field": "productid",
        "table": "products",
        "valuefield": "id",
        "textfield": "product_name"
    }
]

This is safe — the codes section only affects UI dropdown rendering and is ignored if the referenced table doesn't exist yet. Do NOT defer adding codes entries until the referenced module is built.

pyproject.toml Configuration

Configure pyproject.toml with the following guidelines:

• Package name: Must match the module directory name exactly (e.g., name = "customer_management")
• Dependencies: Only declare direct code dependencies, NOT framework packages
  • Use "sqlor" (NOT "sqlor-database-module")
  • Use "bricks_for_python" (NOT "bricks-framework")
  • Do NOT include foundation packages (ahserver, appbase, rbac, apppublic) — these are installed by build.sh separately and are not on PyPI
  • Do NOT include ahserver — it is installed as a core dependency by build.sh
• pip install . must work locally: After build.sh installs all core dependencies, pip install . should succeed without trying to fetch non-PyPI packages

Example:

[build-system]
requires = ["setuptools>=45", "wheel"]
build-backend = "setuptools.build_meta"

[project]
name = "mymodule"
version = "1.0.0"
description = "Module description"
requires-python = ">=3.8"
dependencies = [
    "sqlor",
    "bricks_for_python",
]

[tool.setuptools.packages.find]
where = ["."]
include = ["mymodule*"]

Project Files

• pyproject.toml: Python packaging configuration matching module name
• README.md: Comprehensive module documentation — must describe the module's purpose, features, data tables, installation, and integration in plain language. Never leave it empty or boilerplate-only.
• skill/SKILL.md: MANDATORY agent-facing reference — YAML frontmatter (name, description) + markdown body covering: module architecture, data model, key DSPY endpoints, and pitfalls. This is the file AI agents read to understand how to work with the module. Every independent module repo must have this file.
• build.sh: Module build script that should be integrated into the main application's build process

skill/SKILL.md Content Guidelines

Every module's skill/SKILL.md must contain at minimum:

1. YAML frontmatter: name and description fields
2. Architecture overview: Key directories and files, what each does
3. Data model: Tables + key columns (even for interaction-layer modules, note they have no tables)
4. Key endpoints: DSPY files and what they do
5. Pitfalls: Module-specific traps — wrong API patterns, column name gotchas, DBPools quirks

Keep it concise. This is consumed by AI agents, not humans reading for the first time.

Build Process Integration

When the main application's build.sh runs, it will process all modules (including foundation modules like appbase, rbac, and business modules like contract_management) that have models/ or json/ directories:

1. Install xls2ddl tool in the virtual environment
2. For modules with models/ directory containing .xlsx files: run xls2ddl mysql . > mysql.ddl.sql
3. For modules with models/ directory containing .json files: run json2ddl mysql . > mysql.ddl.sql
4. For modules with json/ directory: run xls2ui -m ../models -o ../wwwroot ${modulename} *.json to generate CRUD UI files
5. Create symbolic links from module wwwroot to main application wwwroot

CRITICAL: Symlink Generated CRUD Directories

The xls2ui tool generates subdirectories in wwwroot/ for each CRUD definition (e.g., wwwroot/product_category_tree/, wwwroot/product_list/). Each directory contains auto-generated index.ui, get_*.dspy, add_*.dspy, update_*.dspy, delete_*.dspy. These directories are NOT covered by the individual-file symlink rules above and MUST be linked separately:

# Link generated CRUD directories (NOT individual files):
cd /path/to/sage/wwwroot/module_name/
ln -sf /path/to/module/wwwroot/crud_alias_name/ .

# Or in build.sh, auto-link all non-api/non-assets directories:
for d in "$MODULE_DIR/wwwroot"/*/; do
    [ -d "$d" ] || continue
    dname=$(basename "$d")
    case "$dname" in api|styles|scripts) continue ;; esac
    ln -sf "$d" "$MODULE_WWWROOT/$dname"
done

Symptoms:

• CRUD pages return 404 even though xls2ui ran successfully — the generated directories exist in the module's wwwroot but aren't linked to Sage's wwwroot.
• ahserver raises processorResource.py: raise Exception(f'{str(request.url)=} invalid path') — this specific error means the URL path is not registered in ahserver's route table, which almost always means the file/directory is not symlinked to the deployment target's wwwroot.

CRITICAL: CRUD-Generated Directories Are READ-ONLY — Never Modify Files Inside Them

Hard rule: The xls2ui tool generates subdirectories in wwwroot/ (e.g., wwwroot/product_category_tree/, wwwroot/product_list/). Every file inside these directories — index.ui, get_*.dspy, add_*.dspy, update_*.dspy, delete_*.dspy — is auto-generated and must NEVER be edited directly.

JSON manipulation safety — use json.load/dump, not string insertion: Complex .ui files (deeply nested JSON) are notoriously error-prone to modify via string insertion. A single missing comma or bracket silently breaks the page. Always prefer: read with json.load() → modify the Python dict → write with json.dump(). The json module guarantees valid output; string replacement does not.

Pitfall: xls2ddl data_new_tmpl uses {{summary[0].pkey}} → WHERE None = %s In xls2ddl commit fc91486 (June 2026), the data_new_tmpl was updated to add a post-INSERT SELECT to return user_data for tree widgets. It used {{summary[0].pkey}} but model JSONs use primary (an array), NOT pkey. Jinja2 resolves the missing attribute to Python None, which renders as ${None}$ in the generated SQL. sqlor then produces WHERE None = %s, causing OperationalError: (1054, "Unknown column 'None' in 'WHERE'") on every add operation across all CRUD tables.

Fix: Change {{summary[0].pkey}} to {{summary[0].primary[0]}} in xls2ddl/xls2ddl/tmpls.py line 308.

To change behavior of CRUD-generated files, you MUST:

1. Modify the source configuration: json/{alias}.json (CRUD definition) or models/{table}.json (table definition)
2. Re-run build.sh (which calls xls2ui) to regenerate the directory
3. The regenerated files overwrite whatever was there

When json/ config cannot express the needed logic:

1. Create a custom dspy in wwwroot/api/ (e.g., api/add_user.dspy, api/update_user.dspy)
2. Point to it from json/{alias}.json via new_data_url / update_data_url / delete_data_url:

   "params": {
       "new_data_url": "{{entire_url('/rbac/api/add_user.dspy')}}",
       "update_data_url": "{{entire_url('/rbac/api/update_user.dspy')}}"
   }
   

3. Register the api paths in scripts/load_path.py (RBAC) and sage/load_path.py (routing)
4. Re-run xls2ui to regenerate — the generated index.ui will use the custom URLs

Pitfall: bricks Tabular code fields send values as lists — unwrap before sor.C/U

When a CRUD JSON config uses alters with code lookups (e.g., supplier_id → suppliers table), the bricks Tabular form may submit the field value as a list ['val1', 'val1'] instead of a plain string.

Root cause and correct fix: The list comes from new_data_url including the field as a query param (?supplier_id=...) while the field is also in editexclouded. When the form submits, ahserver merges URL params + form body into params_kw, creating a list. The correct fix is the JSON config: remove the duplicate from new_data_url if the field is in editexclouded. Do NOT add list-unwrap workarounds in every dspy.

Common customizations in api/ dspy:

• Set default values (e.g., ns['created_at'] = curDateString())
• Auto-generate required business codes (e.g., contract_code) — see references/custom-create-dspy-required-fields.md
• Clean _text suffix fields before sor.U/C (Tabular sends these from code lookups)
• Preserve password on update (pop empty password from ns)
• try/except with format_exc() + str(e) to show real errors to frontend

WRONG: Editing wwwroot/product_category_tree/index.ui or any .dspy inside it CORRECT: Editing json/product_category_tree.json, then re-running build.sh; OR creating wwwroot/api/custom.dspy + pointing json config to it

Symptom: User will reject and correct you immediately if you modify files in these directories. These are build artifacts, not source code.

CRITICAL: Model JSON changes require xls2ui rebuild. Changing models/*.json (codes, fields, indexes) does NOT auto-update the generated CRUD .ui files. You MUST re-run build.sh (which calls xls2ui) after ANY model JSON change. Otherwise the filter dropdowns, edit forms, and data views continue using the old model. This is the #1 cause of "I changed the model but the page still shows old data." See references/model-change-rebuild-pitfalls.md for full details including .xlsx/.json load order, get_code.dspy module.table resolution failure, and collation mismatch pitfalls.

Pitfall: model change → rebuild → THREE post-rebuild checks required

After running build.sh for a module, verify THREE things before declaring done:

1. Files exist: ls wwwroot/<module>/<alias>/index.ui — rebuild can fail silently
2. Symlinks intact: if alias-based CRUD URLs use root-level paths (/<alias> instead of /<module>/<alias>), ensure symlinks survive the rebuild: ls -la sage/wwwroot/<alias> → should point to sage/wwwroot/<module>/<alias>
3. RBAC refreshed: new index.ui has new dataurl/get_code.dspy calls → verify permissions exist AND restart Sage (RBAC cache has 10-min TTL; load_path.py only inserts new permissions, doesn't assign roleperm for existing ones)

Pitfall: xls2ui strips code-type fields from generated editexclouded

When a field appears in BOTH browserfields.alters (with uitype: "code" and dataurl) AND editexclouded in the source CRUD JSON config, xls2ui strips it from the generated index.ui's editexclouded list. This causes bricks.js to render a UiCode dropdown widget in the add/edit form, which calls .build_options() on undefined data.

Symptom: TypeError: Cannot read properties of undefined (reading 'length') at bricks.UiCode.build_options when clicking Add/Edit. Stack trace: bricks.Tabular.build_add_form → bricks.InlineForm → new bricks.UiCode → .build_options().

Diagnosis: Compare the source CRUD JSON params.editexclouded against the generated wwwroot/{alias}/index.ui's row_options.editexclouded. Code fields with dataurl in alters will be missing.

Fix: Add the missing field(s) back to editexclouded in the generated index.ui. Since these files are .gitignore'd, use git add -f. Note: the fix is undone if xls2ui re-runs (via build.sh), so deploy with git pull only, skipping build.sh. Long-term: fix xls2ui to not remove fields from editexclouded just because they appear in alters.

Verification script (adapt for the module):

for alias in supply_contracts_list distribution_agreements_list; do
  want=$(python3 -c "import json; print(set(json.load(open('json/${alias}.json'))['params']['editexclouded']))")
  got=$(python3 -c "
import json,re
t=open('wwwroot/${alias}/index.ui').read()
m=re.search(r'\"editexclouded\"\s*:\s*\[(.*?)\]',t,re.DOTALL)
print(set(s.strip().strip('\"') for s in m.group(1).split(',') if s.strip()))
")
  python3 -c "missing=${want} - ${got}; assert not missing, f'${alias}: {missing}'"
done && echo "OK"

Pitfall: cheight on CRUD index.ui causes title row to fill viewport

Generated wwwroot/{alias}/index.ui files contain "cheight":40 on both the outer VBox and inner Tabular options. This hardcodes a character-height value that causes the title/header row to expand and fill the entire screen, obscuring the data table.

Symptom: CRUD page title bar takes up full viewport height; data table invisible or pushed far below.

Fix: Remove ALL cheight properties from the generated index.ui. Tabular handles its own height automatically:

- "options":{"cheight":40,"width":"100%"},
+ "options":{"width":"100%"},

And inside the Tabular options:

  "options":{
      "width":"100%",
-     "cheight":40,

Affected files: Every wwwroot/{alias}/index.ui generated by xls2ui. Check all aliases:

grep -rn 'cheight' wwwroot/*/index.ui

CRITICAL: Do NOT Replace Framework-Generated Code with Custom Scripts

Core principle: The CRUD framework and its auto-generated endpoints are base functionality — stable, tested, and maintained. Do NOT replace them with hand-written scripts unless there is a genuine special requirement that the framework cannot handle.

Wrong approach: Creating wwwroot/api/llm_list.dspy to replace the auto-generated get_llm.dspy just to add _text fields for display.

Correct approach: Use the framework auto-generated endpoints, and solve display issues through proper configuration (e.g., fix the dataurl API to return [{field_name, field_name_text}] format).

Why this matters:

• Base framework endpoints handle RBAC, logined_userorgid, confidential_fields redaction, and DBFilter parsing correctly
• Custom scripts often violate .dspy file conventions (imports, dict access patterns, etc.)
• Replacing stable framework code with custom scripts creates maintenance burden — "today this way, tomorrow that way" makes the system unmaintainable

When to propose custom scripts: Only when there is a genuine special requirement that the CRUD framework cannot handle. Even then, discuss the approach first before implementing — get confirmation that the deviation is necessary and the proposed solution is acceptable. The user has zero tolerance for unsolicited replacements of stable framework code.

CRITICAL: Hand-written list .dspy SHADOWS framework auto-generated endpoints

When a module has CRUD JSON definitions in json/ (e.g., table_list.json), the Sage CRUD framework (via xls2ui during build.sh) auto-generates list endpoints inside subdirectories (e.g., wwwroot/table_list/get_table_list.dspy). If you also create a hand-written wwwroot/api/get_{table}_list.dspy, it shadows (overrides) the framework-generated one. This causes:

• 500 errors: hand-written code may apply filters (e.g., org_id) on fields that don't exist in the table
• 403 errors: hand-written files bypass the framework's built-in RBAC and logined_userorgid handling
• Silent data leakage: hand-written code misses confidential_fields redaction

Rule: If json/{alias}_list.json exists, do NOT create wwwroot/api/get_{table}_list.dspy. Only create hand-written .dspy for:

• Custom business logic endpoints (create, update, delete actions beyond simple CRUD)
• Client-specific dropdown data APIs that return [{value, text}] format (these are NOT list endpoints — they serve Form dropdowns)

How to detect: If a CRUD list endpoint returns 500 or 403, check whether both a json/ definition AND a hand-written api/get_*_list.dspy exist for the same table. If both exist → DELETE the hand-written one.

CRITICAL: CRUD-Generated wwwroot Directories Must NOT Be Git-Tracked

The xls2ui tool generates subdirectories in wwwroot/ for each CRUD definition (e.g., wwwroot/product_list/, wwwroot/llm/). Each contains index.ui, get_*.dspy, add_*.dspy, etc. These are auto-generated build artifacts and must NOT be committed to git.

Required .gitignore entries for any module using CRUD:

# CRUD definition directories (auto-generated by Sage platform)
wwwroot/llm/
wwwroot/llmusage/
wwwroot/llm_api_map/
# ... etc for each CRUD alias

WRONG: git add wwwroot/ — this pulls in auto-generated CRUD directories CORRECT: git add wwwroot/index.ui wwwroot/api/ wwwroot/*.js wwwroot/*.css — add only hand-written files

Symptom: User will reject or correct you if you commit CRUD-generated directories. These files are regenerated by build.sh on every deployment.

CRITICAL: Selective git add in Module Repos

Never use git add -A or git add wwwroot/ in Sage module repos. Module repos have multiple categories of generated content that must NOT be committed:

• build/ and *.egg-info/ — Python build artifacts
• __pycache__/ — Python bytecode cache
• CRUD-generated wwwroot subdirectories — auto-generated by xls2ui
• *.swp, *.swo — vim/editor swap files (add to .gitignore)

Always use selective git add:

# For Python package changes:
git add mymodule/__init__.py mymodule/init.py

# For frontend changes (only hand-written files):
git add wwwroot/index.ui wwwroot/api/ wwwroot/*.js wwwroot/*.css

# For CRUD JSON changes:
git add json/ models/

CRITICAL: Manual Symlink for New Files During Development

When you create new .ui, .dspy, .js, or .css files in a module's wwwroot/ during development, they are NOT automatically available until linked:

# Link individual files:
cd /path/to/sage/wwwroot/module_name/
ln -sf /path/to/module/wwwroot/new_page.ui .
ln -sf /path/to/module/wwwroot/api/new_api.dspy api/
ln -sf /path/to/module/wwwroot/new_script.js .

# Or re-run build.sh to refresh all links

IMPORTANT: .js and .css files must be linked at the module wwwroot root level (not in subdirectories like scripts/ or styles/). Sage's header.tmpl only scans for JS/CSS at the root of each linked wwwroot directory.

Symptom: 500 error with fpath is None and invalid path in server logs — the file exists in the module but isn't linked to Sage's wwwroot.

CRITICAL: build.sh SAGE_ROOT detection

When writing build.sh for a standalone module repo (e.g. ~/repos/mymodule/), the script is NOT nested under Sage's directory. Do NOT assume $SCRIPT_DIR/../.. points to Sage root. Use a search pattern:

for candidate in "$SCRIPT_DIR/../.." "$HOME/repos/sage" "$HOME/sage"; do
    if [ -d "$candidate/wwwroot" ] && [ -d "$candidate/py3/bin" ]; then
        SAGE_ROOT="$(cd "$candidate" && pwd)"
        break
    fi
done

Sage System Integration

New modules must be wired into the Sage system in FOUR places before they appear in the UI.

1. app/sage.py — Module Loading

Add the import at the top of ~/repos/sage/app/sage.py (near other module imports):

from mymodule.init import load_mymodule

Add the load call in the init() function (near other load calls):

load_mymodule()

2. build.sh — Installation Loop

Add the module name to the installation loop in ~/repos/sage/build.sh:

for m in appbase rbac ... existing_modules mymodule
do
    echo "install $m module..."
    cd $cdir/pkgs
    git clone https://git.opencomputing.cn/yumoqing/$m
    cd $m
    $cdir/py3/bin/pip install .
    ...
done

3. load_path.py — RBAC Permissions

Add entries to the paths="" string in ~/repos/sage/load_path.py:

/mymodule logined
/mymodule/index.ui logined
/mymodule/menu.ui any
/mymodule/api/endpoint.dspy logined
/mymodule/scripts/file.js any

Permission rules:

• any — no login required. Use for: menu.ui, static JS/CSS files, public resources.
• logined — requires authenticated user. Use for: .ui pages, .dspy APIs.
• owner.superuser / owner.operator — role-restricted admin features.

After editing load_path.py, run: cd ~/repos/sage && ./py3/bin/python load_path.py

Note: load_path.py is the centralized declarative source of truth. set_role_perm.py is for ad-hoc fixes only and does NOT survive a fresh database restore.

See references/independent-web-app-pattern.md for building standalone ahserver web applications (own venv, own port, own config.json) — distinct from Sage sub-modules. See references/rbac-init-pattern.md for bootstrapping organization types, roles, users, and role-permission assignments for independent apps. See references/read-only-module-pattern.md for the complete read-only module template. See references/test-env-module-deployment.md for deploying modules from ~/test/ into Sage test environment (pip install, symlink, RBAC, restart, git push). See references/public-endpoint-cron-pattern.md for exposing dspy endpoints without auth for cron/background jobs. See references/hub-dashboard-pattern.md for the hub entry page pattern (stat cards + nav buttons + urlwidget content area) for business modules with CRUD + management views. See references/registry-table-pattern.md for the registry/mapping table + dynamic table routing pattern. See references/llmage-openai-endpoint-pattern.md for the pattern of adding OpenAI-compatible API endpoints to the llmage module (/v1/chat/completions, /v1/video/generations, /v1/image/generations). See references/llmage-data-model.md for the llmage data model key tables, JOIN patterns, which fields belong to which table (apiname is in llm_api_map NOT llm), and subquery alias scoping pitfalls. See references/api-integration-pattern.md for the uapi/llmage 4-table API integration pattern (no code changes needed). See references/tencent-hunyuan-api.md for Tencent Hunyuan model specifications, pricing, and Sage integration notes. See references/external-api-integration-module-pattern.md for modules that integrate with third-party APIs. See harnessed-module-development skill → references/vendor-callback-pattern.md for vendor webhook/callback endpoint implementation. See references/rbac-tables.md for the full RBAC table chain (organization→role→userrole→rolepermission→permission) and the correct init order for org/role/user creation in independent apps. See references/volcengine-ark-api.md for Volcengine Ark real person portrait asset API — AK/SK signing, asset group/asset workflow, and hybrid module pattern (own client + uapi config). See references/downapp-proxy-api-pattern.md for the Downapp user proxy API pattern — centralized vendor config table, org-to-resource mapping table, ownership validation, and multi-vendor factory design. See references/configurable-rule-engine-pattern.md for the registry-based dynamic rule engine pattern — when modules need user-configurable validation rules (discounts, permissions, workflows). See references/platform-cross-org-table-pattern.md for cross-org (platform-level) table design — tables that bridge multiple organizations without resellerid isolation, unified supplier mapping (internal + external), and menu grouping with separators. See references/cache-control-pattern.md for the config.json module_cache toggle pattern — enable/disable in-memory caching per module via config.json. See references/sage-deployment-pitfalls.md for site-packages sync, DSPY async limitations, CRUD before_insert hooks, and browser testing requirement. See references/pipeline-app-deployment-pitfalls.md for pipeline-app umbrella repo recovery, bricks dist build, dspy return-outside-async-with, and RBAC static file permissions. See references/feature-enablement-three-layer-pattern.md for systematically unlocking features gated behind hardcoded UI/dspy/backend limits. See references/cross-database-query-pattern.md for the cross-database JOIN pattern — when a .dspy queries tables from another module's database using fully-qualified names (e.g., product_management.product from the discount database). See references/social-platform-pattern.md for content/social module patterns — counter updates (R+U), toggle likes, nested comments (parent_id+reply_count), view counting, paid downloads, media type classification, and paginated feed APIs. See references/ai-compute-service-pattern.md for deploying GPU-accelerated AI services (4090 server → ahserver HTTP API with nginx IP whitelist → Sage llmage+uapi integration → Hermes skill consumption).

• IntegrityError handling: See references/dspy-error-handling.md for error handling, UPSERT, and form id management patterns in .dspy. See references/dspy-nonetype-defense.md for mandatory try/except wrapper patterns on all DSPY-called functions.

See references/dspy-pitfalls.md for return format, sqlor API signatures, param cleaning patterns, return-inside-async-with pitfalls, dbname naming, and $placeholder escaping.

See references/module-extraction-pattern.md for extracting embedded modules from umbrella repos to independent repos with git subtree split.

See references/bricks-dist-build.md for the mandatory bricks dist/ build + symlink step after fresh clone/restore. See references/crud-list-query-optimization-pattern.md for optimizing slow CRUD list queries — exclude TEXT/LONGTEXT fields from SELECT, add composite indexes on (filter_field, sort_field). See scripts/dspy_audit.sh for the pre-commit dspy audit script — run before any commit touching .dspy files. See scripts/module_compliance_audit.py for the batch module compliance audit — checks init/data.json format, load_path.py existence, and model codes parentid= compliance across multiple module repos. See references/sage-config-and-db-export.md for parsing Sage's non-standard config.json (bare objects), RC4 password decryption, table discovery from models/*.json, and mysqldump export patterns. See references/i18n-merge-pattern.md for the i18n merge workflow — consolidating module translations into wwwroot/i18n/ via scripts/merge_i18n.py (updated: uses build.sh module list, ~/repos paths).

See references/popup-form-refresh-pattern.md for the pattern of refreshing parent page content after a Form inside a PopupWindow submits successfully (target: app.sage_main_content). See references/bricks-tabular-code-field-pattern.md for bricks Tabular code field requirements — static data arrays vs dynamic dataurl+valueField/textField patterns, and common undefined.length errors. See references/feature-page-addition-checklist.md for the 7-step workflow when adding a new feature page to an existing Sage module (dspy → ui → menu → RBAC → CSS → commit per repo → deploy). See references/supplychain-attachment-pattern.md for filemgr attachment integration with drag-drop Form upload pattern (Form submit_url + UiFile multiple). See references/marketing-discount-setting.md for marketing plan product discount override with base-comparison validation and flag pattern.

See references/toolbar-ensure-pattern.md for the one-click toolbar check-or-create pattern — actiontype: "dspy" + params_mapping to ensure a related record exists.

See references/subtable-fk-auto-populate.md for the pattern of auto-populating foreign keys in subtable add forms (editexclouded + new_data_url query params).

Multi-Tenant Reseller Module Pattern

When different resellers (organizations) need their own independent data structures (category trees, product definitions, configs), use this org_id-scoped isolation pattern:

Core Principle

Every table, CRUD definition, and API endpoint must be scoped by org_id. Different resellers have completely independent data — they never share categories, products, or configs.

Table Design

• Every table MUST have org_id VARCHAR(32) DEFAULT '0' field
• Composite unique indexes use (org_id, business_key) — e.g., UNIQUE(org_id, product_code)
• Foreign key lookups must include org_id in JOIN conditions: JOIN t2 ON t1.cat_id = t2.id AND t1.org_id = t2.org_id

Dynamic Attributes: product_type + extra_json (NOT physical table routing)

• Use a product_type field as a string identifier (not a physical table name)
• Store reseller-specific product attributes in extra_json (TEXT/LONGTEXT column)
• Each reseller defines their own structure inside extra_json — no schema changes needed
• Category tables store product_type and product_type_title for display, not product_table_name
• Standardized APIs (product_detail, product_use) parse extra_json and return extra_parsed

CRUD Definition Requirements

• logined_userorgid: "org_id" on ALL list CRUD definitions — enables automatic org-scoped filtering
• Category dropdown dataurl must point to an API that filters by current user's org_id
• subtables[].url references must use ../ prefix and respect org boundaries

.dspy API Requirements

• Get org_id: use await get_userorgid() (registered global async function by ahserver processorResource.py) — do NOT use ServerEnv() in .dspy files
  • Correct: org_id = (await get_userorgid()) or '0'
  • Correct (with optional param override): org_id = params_kw.get('org_id', None) or (await get_userorgid()) or '0'
  • WRONG: env = ServerEnv(); org_id = getattr(env, 'orgid', None) ...
• ALL SQL queries MUST include org_id in WHERE clauses
• Create/update/delete operations MUST verify the record belongs to current org before modifying
• Parent-child relationships (category trees) MUST verify parent belongs to same org
• Category dropdown APIs (category_options.dspy) must filter: WHERE org_id = ${org_id}$

Operator Configuration Pattern

• product_type_config table: (org_id, operator_id, category_id, config_name) unique key
• Allows each operator within a reseller to define their own config for each category
• config_json TEXT field stores flexible configuration; parse to config_parsed on retrieval
• Query with fallback: WHERE operator_id = ${user_id}$ OR operator_id = '0' (0 = global default)

Pitfalls

• Do NOT use product_table_name pointing to physical tables — this breaks when different resellers have different structures. Use product_type (identifier) + extra_json instead.
• Do NOT preset global category data in init/data.json — categories are managed per reseller. Init data should only contain appcodes/encoding definitions.
• JOIN without org_id = data leak — LEFT JOIN product_category pc ON p.category_id = pc.id without AND p.org_id = pc.org_id will return categories from other resellers if IDs happen to overlap.
• Unique index without org_id = cross-reseller conflicts — UNIQUE(product_code) prevents two resellers from using the same code. Must be UNIQUE(org_id, product_code).
• DBPools() is a singleton (@SingletonDecorator) in Python module code — calling DBPools() returns the already-configured instance from startup. Do NOT write db.databases = config.databases or DBPools(config.databases). Simply use DBPools().sqlorContext(dbname).
• In Python module files (*.py), get org_id via ServerEnv(): env = ServerEnv(); org_id = getattr(env, 'orgid', None) or getattr(env, 'org_id', '0'). This is different from .dspy files which use await get_userorgid().

Multi-Module Changes: PR Isolation

When making changes across multiple Sage modules (UI redesign, stat cards, bug fixes, etc.), each module must use a separate feature branch and independent PR. Do NOT combine multi-module changes into a single branch.

Correct pattern:

accounting:     feat/dataviz-accounting  → PR #1
llmage:         feat/dataviz-llmage      → PR #2
dashboard:      feat/dataviz-users       → PR #3

Wrong: One branch feat/all-changes touching accounting/, llmage/, dashboard/ repos.

This keeps review focused, allows partial merges, and prevents blocking unrelated modules on a single review cycle.

Cross-Module Changes: Where to Make Changes

All modules are independent git repos at ~/repos/<module>/. Make changes and commit directly there. There is no sage/pkgs/ directory — it was deleted in May 2026.

# CORRECT: modify in the module's own repo
cd ~/repos/uapi/
# ... make changes ...
git add ... && git commit -m "..." && git push

For cross-module changes (e.g., llmage modifying JOINs that involve uapi tables), check each affected module repo separately. After every round of changes, verify no stale patterns remain:

grep -rn 'old_pattern' ~/repos/<module>/ --include='*.py' --include='*.dspy'

Restart Sage after all code changes to reload modules from disk.

Database Schema Migration

See references/database-migration-pattern.md for the complete workflow when modifying database schemas in production modules:

• Adding new tables / changing relationships (1:N → M:N)
• Writing idempotent migration scripts with safety checks
• Setting RBAC permissions for new CRUD endpoints
• Safe column removal procedures

Intermediate Table Removal

When removing an intermediate/junction table (e.g., uapiset between upapp and uapi), the process requires:

1. Add direct join column: ALTER TABLE child ADD COLUMN parent_id ...
2. Move fields to parent: Move any fields that were on the intermediate table (e.g., auth_apiname) to the parent table (upapp)
3. Update model JSON: Remove old FK field, add new FK field, update indexes, remove codes references to the removed table
4. Update ALL Python code: Remove JOINs to the intermediate table in every .py file. Check the module repo at ~/repos/{mod}/ and any other modules that reference the table.
5. Delete obsolete files: Remove models/{intermediate}.json and json/{intermediate}.json
6. Data migration: Handle shared records (multiple parents sharing same children through the intermediate table). See ~/repos/uapi/scripts/migrate_uapi_upappid.py for the pattern.
7. Verify: grep -rn 'intermediate_table' ~/repos/{mod}/ --include='*.py' — should return nothing.

See references/intermediate-table-removal-pattern.md for the complete detailed pattern with SQL examples. See references/uapi-uapiset-removal-detail.md for a concrete session example (uapiset removal from uapi module).

RBAC Permission Setup

Preferred approach: Per-module scripts/load_path.py

Each business module should own its own scripts/load_path.py that registers all its RBAC paths. This keeps permissions self-contained and auditable per module.

Maintenance rule: 每次代码变更如有新 path 出现，需同步更新此脚本

The script should:

1. Find the Sage root directory automatically
2. Define paths by role tier (any, logined, role-specific)
3. Call set_role_perm.py for each path

See references/per-module-rbac-pattern.md for the complete script template. See references/per-module-load-path-pattern.md for a concrete script template with role-tier separation.

NOTE: Some modules (like llmage) use the central load_path.py in ~/repos/sage/load_path.py instead of a per-module script. Before creating a new scripts/load_path.py, check if the module already has one or if it uses the central approach.

CRITICAL: load_path.py wildcards are FORBIDDEN

NEVER use % or * wildcards in any load_path.py file. This is a hard policy rule, not a reliability suggestion. All RBAC paths must be listed explicitly — every .dspy file, every CRUD subdir file, every .ui file, every image.

Correct approach:

PATHS_LOGINED = [
    f"/{MOD}/api/endpoint1.dspy",
    f"/{MOD}/api/endpoint2.dspy",
    f"/{MOD}/api/get_pricing_display.dspy",
    # ... list every .dspy file explicitly
    f"/{MOD}/table_alias",
    f"/{MOD}/table_alias/index.ui",
    f"/{MOD}/table_alias/get_table_alias.dspy",
    f"/{MOD}/table_alias/add_table_alias.dspy",
    f"/{MOD}/table_alias/update_table_alias.dspy",
    f"/{MOD}/table_alias/delete_table_alias.dspy",
]

CRUD subdirectory standard pattern (5 files per alias, generated by xls2ui from json/*.json):

f"/{MOD}/{alias}",
f"/{MOD}/{alias}/index.ui",
f"/{MOD}/{alias}/get_{alias}.dspy",
f"/{MOD}/{alias}/add_{alias}.dspy",
f"/{MOD}/{alias}/update_{alias}.dspy",
f"/{MOD}/{alias}/delete_{alias}.dspy",

Maintenance rule: When adding new .dspy files, you MUST also add them to PATHS_LOGINED (or PATHS_ANY) explicitly. When removing files, remove the corresponding entries.

Cross-module audit: When fixing wildcards in one module, check ALL modules (find ~/repos -name "load_path.py") for remaining wildcards with grep -rn '%' --include="load_path.py". All 9+ modules should be wildcard-free.

4. global_menu.ui — Menu Entry

Add a menu item to sage/wwwroot/global_menu.ui inside the items array:

,{
    "name": "modulename",
    "label": "模块名称",
    "icon": "fa fa-icon-name",
    "url": "{{entire_url('/modulename/index.ui')}}",
    "target": "app.sage_main_content"
}

CRITICAL: Do NOT wrap module menu items in {% if get_user() %} or role conditionals. Sage's dynamic menu loading does not work — items inside conditionals are never rendered. Place all module items unconditionally. RBAC (load_path.py) handles access control, not menu visibility.

Usage:

cd ~/repos/sage
./py3/bin/python ~/repos/<module>/scripts/load_path.py

The script must register both directory paths and file paths for CRUD aliases:

# Directory path (auto-matches index.ui)
set_role_perm.py "logined" "/module/crud_alias"
# File path (direct access)
set_role_perm.py "logined" "/module/crud_alias/index.ui"

See references/per-module-rbac-pattern.md for the complete script template. See references/per-module-load-path-pattern.md for a concrete script template with role-tier separation.

Legacy approach: Central load_path.py

Add entries to the paths="" string in ~/repos/sage/load_path.py:

/module_name logined
/module_name/index.ui logined
/module_name/menu.ui any
/module_name/api/xxx.dspy logined
/module_name/xxx.js any

• logined — pages and API endpoints that require authentication
• any — static resources (menu.ui, .js, .css, images) that must load without auth

Steps

1. Edit ~/repos/sage/load_path.py — add entries to the paths string in the appropriate location (alphabetically by module, or grouped by feature)
2. Run on the server: cd ~/repos/sage && ./py3/bin/python load_path.py
3. Restart Sage: ./stop.sh && ./start.sh

Note: set_role_perm.py is for ad-hoc fixes only and does NOT survive a fresh database restore. Both set_role_perm.py and load_path.py must use explicit paths — no wildcards in either.

IMPORTANT: Do NOT manually reference .css/.js files in .ui widgets with <link> or <script> tags. ahserver auto-serves these files from wwwroot/. However, they still need RBAC registration in load_path.py (see references/per-module-rbac-pattern.md).

Integrating a New Module into Sage

Beyond pip install and wwwroot symlinks, a new business module requires FOUR changes in the Sage repo:

1. app/sage.py — Add import and load call:

   from modulename.init import load_modulename
   # ... in init():
   load_modulename()
   

2. build.sh — Add module name to the installation loop:

   for m in ... existing_modules modulename
   

3. load_path.py — Add RBAC permission entries (see RBAC Permission Setup section above)

4. global_menu.ui — Add menu entry in sage/wwwroot/global_menu.ui:

   ,{
       "name": "modulename",
       "label": "模块名称",
       "icon": "fa fa-icon-name",
       "url": "{{entire_url('/modulename/index.ui')}}",
       "target": "app.sage_main_content"
   }
   

All four are required. Missing any one causes: (1) module functions not registered, (2) module not installed on fresh deployments, (3) 403 Forbidden on page access, (4) menu item not visible.

5. Execute per-module scripts/load_path.py (if it exists)

If the module has its own scripts/load_path.py, run it to register permissions via set_role_perm.py:

cd ~/repos/sage
./py3/bin/python ~/repos/<module>/scripts/load_path.py

CRITICAL: Dual-layer RBAC pitfall. Some modules (cpcc, product_management, pricing, etc.) have per-module scripts/load_path.py files that call set_role_perm.py directly. However, set_role_perm.py has been failing due to ModuleNotFoundError: No module named 'appPublic.event_dispatcher' in the virtual environment. This means per-module permissions may never reach the database.

Therefore, always ALSO add entries to the central ~/repos/sage/load_path.py. The central load_path.py uses a different code path and is the reliable fallback. If only the per-module script exists and it fails silently, the module will show in the menu but every click returns 403.

Verification: After running load_path.py, confirm permissions exist in the database:

grep -c '/modulename' ~/repos/sage/load_path.py  # should be > 0

CRITICAL: global_menu.ui Menu Items — NO Conditional Blocks

Do NOT place module menu items inside {% if get_user() %} or role-check conditionals. Sage's dynamic menu loading logic does not work — items inside conditionals are never rendered. All business module menu items must be placed at the top level of the items array, unconditionally.

The only exception: the dashboard item (always visible by design) and role-restricted items that the user explicitly wants hidden from certain roles. When in doubt, make it unconditional.

WRONG:

{% if get_user() %}
,{ "name": "mymodule", "label": "我的模块", ... }
{% endif %}

CORRECT:

,{ "name": "mymodule", "label": "我的模块", ... }

Access control is enforced by RBAC (load_path.py), not by hiding menu items. If a user clicks a module they don't have permission for, they get a 403 — which is the correct behavior.

Development Workflow

CRITICAL: Code Modification Workflow — Repos First, Test Second

All code modifications MUST follow this sequence:

1. Modify source code in ~/repos/ directory first — this is the canonical source
2. Git commit immediately in the repos directory
3. Sync to test environment (~/test/ or deployment directory) for verification
4. Never modify test environment directly — changes in ~/test/ are lost and untracked

Why this matters:

• ~/repos/ is version-controlled — changes survive and can be reviewed
• ~/test/ is ephemeral deployment — direct edits there vanish on redeploy
• User has zero tolerance for "I fixed it in test but didn't update repos"

CRITICAL: scp/sed directly to servers is FORBIDDEN. Never use scp to copy files or sed to edit files directly on deployment servers (tokentest, production). Always go through the git workflow: local repos → commit → push → server git pull → pip install. Direct edits bypass version control, create unreviewable changes, and will be overwritten on the next deploy. The one exception is one-off diagnostic commands (grep, mysql queries, log inspection) that don't modify files.

Wrong:

# Modifying test directly
cd ~/test/pipeline-app
# ... make changes ...
# (changes lost on next deploy, no git history)

Also wrong:

# scp/sed directly to server
scp local_file apitest@server:/remote/path/
ssh apitest@server "sed -i 's/old/new/' /remote/file"

Correct:

# Modify repos first
cd ~/repos/pipeline
# ... make changes ...
git add scripts/load_path.py
git commit -m "fix: remove hardcoded Sage paths"
git push

# Then sync to test
cp ~/repos/pipeline/scripts/load_path.py ~/test/pipeline-app/pipeline/scripts/

See references/umbrella-repo-module-extraction.md for extracting embedded modules from umbrella (monorepo) apps into independent repos with preserved git history.

See references/bulk-module-creation-pattern.md for the complete ordered creation sequence, bulk .dspy API generation script, init.py function templates, auto-generated code patterns, cascade delete patterns, and CRITICAL subagent validation checklist. See references/app-level-wwwroot-pattern.md for the critical rule that applications must have a unified app-level wwwroot/ directory — all frontend files go there, NOT scattered across module directories. Modules keep their Python code but move static files to the app's wwwroot.

1. Create module directory structure with all required subdirectories
2. Create table definitions in models/ directory
3. Create CRUD definitions in json/ directory
4. Implement backend functions in {module}/init.py — one async function per CRUD op per table, registered via load_{module}() with ServerEnv
5. Generate wwwroot/api/*.dspy files — thin wrappers delegating to init.py functions (use bulk generation script from reference)
6. Develop frontend using bricks-framework in wwwroot/ (index.ui + menu.ui first, CRUD UI auto-generated by xls2ui)
7. Add initialization data to init/data.json if needed
8. Configure pyproject.toml and write build.sh
9. Write README.md with Sage integration steps
10. Wire module into Sage system: app/sage.py (import + load), build.sh (install loop), load_path.py (RBAC permissions), global_menu.ui (menu entry — unconditional, NOT inside {% if %}), execute per-module scripts/load_path.py if it exists — see Sage System Integration section
11. Create skill/ directory with SKILL.md, references/, assets/, and scripts/ subdirectories

Parameter Access: .py vs .dspy/.ui

Rule: Parameter access differs between Python functions and dspy/ui templates.

In .dspy and .ui templates

Use params_kw directly — it is injected as a global:

# .dspy file
discountid = params_kw.get('discountid')

<!-- .ui file -->
{{params_kw.discountid}}

In Python (.py) functions called from Jinja2

Use request._run_ns.params_kw — the request object passed to ServerEnv-registered functions holds params in its _run_ns namespace:

async def my_func(request):
    env = request._run_ns
    val = (request._run_ns.params_kw or {}).get('key', '')

Pitfall — wrong attribute names. Do NOT use:

• request._run_env → AttributeError (does not exist, correct name is _run_ns)
• request._run_ns.param_kw (singular param) → wrong, the attribute is params_kw (plural params)
• request._params_kw → may be empty at Jinja2 render time
• request.rel_url.query or request.query → unnecessary, params already parsed into _run_ns.params_kw

sqlExe: No Hardcoded ORDER BY / LIMIT / OFFSET in SQL

Rule: Do NOT hardcode ORDER BY, LIMIT, or OFFSET in SQL strings. sqlor handles these via sqlPaging():

# WRONG — hardcoded ORDER BY, LIMIT, OFFSET
sql = "SELECT * FROM t ORDER BY name LIMIT ${pagerows}$ OFFSET ${offset}$"

# CORRECT — use sor.sqlPaging() which handles pagination automatically
ns = {"page": 1, "sort": "name", "order": "desc", "rows": 20}
recs = await sor.sqlPaging("SELECT * FROM t", ns)
# Returns {"total": N, "rows": [...]}

Pitfall: sqlExe returns DictObject — NOT dict-convertible sqlor's sqlExe() returns DictObject instances. dict(r) fails with ValueError: dictionary update sequence element #0 has length 1; 2 is required. Access attributes directly:

# WRONG — DictObject not iterable as (k,v) pairs
rows = [dict(r) for r in recs]

# CORRECT — access attributes individually
rows = [{"id": r.id, "name": r.name} for r in recs]

dspy Database Access: get_sor_context NOT DBPools().sqlorContext

Rule: In .dspy files, use get_sor_context(env, 'module_name') which routes to the correct database per the application's module_dbname config. Do NOT use DBPools().sqlorContext("hardcoded_db"):

# WRONG — hardcoded db name, tied to one application
db = DBPools()
async with db.sqlorContext("pipeline") as sor:
    ...

# CORRECT — routes to correct db per application config
env = request._run_ns
async with get_sor_context(env, "tenant") as sor:
    ...

This way, get_sor_context(env, "tenant") resolves to pipeline database in pipeline-app and sage database in Sage.

Bricks HBox Column Widths: cwidth vs width%

Critical — cwidth in HBox is NOT proportional. In bricks HBox (plain flex containers), cwidth sets a fixed pixel width: charsize × cwidth (e.g., cwidth=3 ≈ 42px). This is different from Tabular's DataRow where cwidth=10 is the default.

For proportional column widths in HBox, use width with percentage values:

// WRONG — fixed 42px/70px columns, won't scale
{"widgettype": "HBox", "subwidgets": [
    {"widgettype": "Text", "options": {"cwidth": 3}},
    {"widgettype": "Text", "options": {"cwidth": 5}}
]}

// CORRECT — proportional 22%/34% columns
{"widgettype": "HBox", "options": {"width": "100%"}, "subwidgets": [
    {"widgettype": "Text", "options": {"width": "22%"}},
    {"widgettype": "Text", "options": {"width": "34%"}}
]}

This applies to any bricks HBox, including hand-built list rows. Only Tabular's DataRow handles cwidth proportionally.

Git Initialization & First Push for New Modules

When creating a brand-new module repo (not yet on Gitea), follow this exact sequence:

Pitfall: Gitea "Push to create" is DISABLED

New repos MUST be created on the Gitea web UI (git.opencomputing.cn) before git push. SSH push-to-create returns: Gitea: Push to create is not enabled for users. fatal: Could not read from remote repository.

Fix: Create the empty repo on Gitea web UI first, THEN push locally.

Pitfall: Auto-generated README causes rebase conflict

Gitea creates an initial README.md when you create the repo. Your local commit also has a README.md. First git pull --rebase triggers CONFLICT (add/add): Merge conflict in README.md.

Fix:

git checkout --ours README.md    # keep local version
git add README.md
GIT_EDITOR="true" git rebase --continue   # "true" avoids nano/editor hang
git push -u origin main

Pitfall: git rebase --continue opens nano editor

In non-interactive terminal sessions, git rebase --continue opens nano which hangs waiting for stdin. Always prefix with GIT_EDITOR="true".

Standard first-push sequence

cd ~/repos/{module}
# 1. Write .gitignore FIRST (see references/module-gitignore.md for template)
# 2. Init and commit
git init && git add -A && GIT_EDITOR="true" git commit -m "feat: initial commit message"
# 3. Add remote
git remote add origin git@git.opencomputing.cn:yumoqing/{module}.git
# 4. Pull, resolve conflicts, push
git pull origin main --rebase 2>&1
# If conflict: git checkout --ours README.md && git add README.md && GIT_EDITOR="true" git rebase --continue
git push -u origin main

CRITICAL: Never Fabricate Pricing or Config Data

Pricing data (元/次, 元/秒, 元/token, etc.) must come from official vendor sources only. Guessing, estimating, or pulling from outdated references is a production incident. The user explicitly framed this as a "生产事故".

Rule: If the official pricing page/documentation is not accessible (SPA, requires login, etc.), tell the user honestly that you cannot find it. Do NOT:

• Estimate from similar tier models
• Extrapolate from older versions
• Use memorized pricing that may be stale
• Pull from skill references written by earlier sessions (those were also guesses)

How to get pricing: Ask the user for screenshots or text from the official console/pricing page, or direct them to provide the vendor URL with accessible data.

Applies to: SQL pricing values, YAML price_factors, unit_prices in pricing_program_timing, unit_price in P4 migration scripts, and any other config where a wrong number has real billing consequences.

Verification Steps

CRITICAL: Self-Testing Before Commit — MANDATORY

User has zero tolerance for untested code. Before committing any module code, you MUST:

1. Deploy to a test environment — create a venv, install deps, start the app on a test port
2. Exercise every endpoint — curl or browser-test each .dspy API, each .ui page
3. Check server logs — tail -f logs/*.log during testing, catch import errors / AttributeErrors / 500s
4. Cross-validate against working Sage code — find real working .dspy files in ~/repos/sage/wwwroot/ and compare patterns

Wrong approach: Write code → commit → tell user to deploy and test Correct approach: Write code → self-deploy → self-test → fix issues → commit only when verified

If you cannot set up a test environment, say so explicitly and do NOT commit.

CRITICAL: dspy Batch Audit Before Commit — MANDATORY PRE-COMMIT GATE

Run this audit BEFORE every commit that touches .dspy files. This is not optional — it's a hard gate. If violations are found, fix them FIRST, then commit. Do NOT commit and fix later.

Full list of pre-loaded symbols in dspy execution context (available without any import):

• json (json.dumps, json.loads)
• datetime (datetime, date, timedelta)
• time (time.time, etc.)
• os (os.path, etc.)
• debug, exception, error, info, warning, critical (from appPublic.log)
• DBPools (from sqlor.dbpools)
• get_sor_context (from sqlor.dbpools)
• DictObject (from appPublic.dictObject)
• get_user, params_kw, format_exc, password_encode, password_decode
• curDateString, timestampstr, getID
• functools.partial
• ahserver.filestorage.FileStorage
• RegisterFunction (from appPublic.registerfunction)
• DBFilter, ArgsConvert (from sqlor.filter)
• All ServerEnv-registered functions (from every module's load_*())

Module-internal imports MUST go through load_*(): from mymodule.utils import foo in dspy is not allowed. Instead, export foo via load_mymodule() in init.py:

# init.py
from .utils import foo
def load_mymodule():
    env = ServerEnv()
    env.foo = foo

Pre-commit audit commands:

# Detect ALL import statements (only DBFilter is acceptable)
grep -rn "^import\|^from" wwwroot/ --include='*.dspy' | grep -v "sqlor.filter"
# The output should be EMPTY. If any lines appear, those imports must be
# removed (if pre-loaded) or moved to load_*() export (if module-internal).

Workflow enforcement:

1. Write/edit .dspy files
2. Run audit script → fix all violations
3. Re-run audit → confirm zero violations
4. Only then: git add → git commit

Why this matters: User will reject commits with violations. Fixing after commit creates noisy history and loses trust. The audit takes 5 seconds — do it every time.

See references/dspy-batch-fix-pattern.md for the systematic fix approach.

Symptom: return data type error, <class 'NoneType'> for a dspy that clearly has a return statement. Fix: move return OUTSIDE the async with block.

When a 500/NoneType error occurs with a stack trace, the FIRST file to inspect is the one named in the error. Do NOT chase tangential issues before examining the file at the top of the traceback.

Real case: 3 consecutive currency_stats.dspy return data type error, NoneType. Fixed init.py try/except, etl.py, SQL columns — all irrelevant. Root cause: JSON-format DSPY file where Python-script was expected.

Rule: Read the file in the error trace FIRST.

CRITICAL: Database Name Consistency Check

When a module uses its own database (not sage), verify in THREE places:

1. Python module init.py: DBNAME = "actual_db_name" (not module name)
2. App entry global_func.py: get_module_dbname() returns correct DB name
3. App entry cms.py: get_module_dbname() override returns correct DB name

Pitfall: DBNAME = "entcms" when actual DB is "ocai_cms" → all queries fail with table-not-found. Pitfall: global_func.py returns 'sage' → dspy files query wrong database.

Verification Steps

MANDATORY: Test before commit. User will reject untested code.

Before committing ANY code changes, you MUST:

1. Build a test environment (~/test/ or dedicated test venv) — do NOT just read code and assume it works
2. Actually run the code — import the module, call the functions, hit the API endpoints
3. Check all affected files — don't just verify the file you changed, check every file that depends on it
4. Verify dspy patterns — read actual Sage dspy files (e.g., gen_code.dspy, callback dspy files) to confirm your patterns match production

Common pitfalls that MUST be caught before commit:

• async with db.sqlorContext(dbname) as sor: — dspy files MUST use this pattern, not sor = DBPools().sqlorContext(dbname)
• No import statements in dspy files (except from sqlor.filter import DBFilter)
• return instead of print(json.dumps(...))
• getID() not uuid()
• await get_user() not get_user()
• Python module DBNAME constants must match actual database name in config.json
• get_module_dbname() return value must match the app's actual database

If you claim "tested" but didn't actually execute the code, the user WILL find out and you WILL lose trust.

Post-deployment testing is equally mandatory. After deploying, immediately verify with curl + auth token:

# Login to get session cookie
curl -s -c /tmp/cookies.txt -X POST 'http://localhost:9180/rbac/user/login.ui' \
  -H 'Content-Type: application/x-www-form-urlencoded' -d 'username=test&password=test123'
# Test all affected endpoints
for path in /module/page /module/api/endpoint.dspy /imgs/icon.svg; do
  code=$(curl -s -o /dev/null -w '%{http_code}' -b /tmp/cookies.txt "http://localhost:9180$path")
  echo "$code $path"  # must be 200, not 403/500
done

Do NOT tell the user "可以测试了" without first running these checks yourself. The user's time is not for debugging your untested code.

• Module loads correctly via load_{modulename}() function
• Module imported and loaded in app/sage.py
• Module added to build.sh install loop
• RBAC permissions added to load_path.py and loaded via python load_path.py
• Menu item added to sage/wwwroot/global_menu.ui (NOT inside {% if %} conditionals)
• wwwroot symlinked to sage/wwwroot/{modulename}/
• All exposed functions work in frontend scripts
• Database operations follow sqlor specifications
• Frontend renders correctly with bricks-framework
• CRUD operations function as defined
• JSON config files contain editable section with correct new_data_url/update_data_url/delete_data_url
• JSON config file entire_url() references use ../ prefix for same-module CRUD aliases
• Initialization data loads properly
• Package builds successfully with pyproject.toml
• Skill documentation created in skill/ directory with complete SKILL.md
• Reference examples, assets, and supporting scripts organized appropriately