---
name: module-development-spec
version: 1.0.0
description: Standardized development workflow for creating modular components following the specified directory structure, frontend/backend conventions, and database integration patterns.
trigger_conditions:
  - User requests to create a new module named "mymodule" or similar
  - Task involves implementing a component with bricks-framework frontend and sqlor database backend
  - Development follows the documented directory structure with wwwroot, models, json, and init directories
  - Task involves creating read-only/dashboard modules that display data without CRUD (no models/json needed)
---

## Work logs / delivery archive

For production module/application development, update a dated work log before finishing the session. Use the actual current date from the system (do not assume the date from earlier messages) and keep late-night follow-up fixes under their real date or explicitly label them as next-day follow-ups. If no log file exists, create one under an appropriate docs/archive path such as `docs/work-log-YYYY-MM-DD.md`.

A useful log entry should include:
- Scope/background and repository/module name.
- Timeline or commit list for the date.
- Key technical decisions and pitfalls discovered.
- Verification performed and any environment-limited verification that could not run.
- Current branch/commit state.

Do not leave a task as "done" without this archive when the user expects deliverables to be retained for later inspection.


This skill defines the complete workflow for developing standardized modules that integrate with the ahserver ecosystem using bricks-framework for frontend and sqlor for database operations.

## Module Philosophy — Host-Agnostic

**Modules are NOT tied to any specific application.** A module is a self-contained unit that any host can load:
- pipeline-app loads it → it's pipeline-app's module
- sage loads it → it's sage's module
- Any future application loads it → it's that application's module

**Core principle**: A module only depends on:
1. Foundation packages (sqlor, ahserver for ServerEnv, appPublic for utilities)
2. Its own data tables (or shared tables like sage's appcodes)
3. Other modules it explicitly imports

A module does NOT depend on:
- Any specific host application's entry point
- Host-specific configuration files
- Host-specific wwwroot paths

The `load_{module}()` function is the ONLY integration point. It registers functions to ServerEnv. The host decides how to wire it in.

**Interaction-layer modules** (like pipeline-task) have NO data tables — they are pure thin wrappers that call other modules' functions via ServerEnv. They provide .dspy + .ui files for user interaction only.

## Directory Structure
```
mymodule/                 # Main module directory (replace mymodule with actual name)
├── mymodule/             # Python package directory
│   ├── __init__.py       # Required Python package file
│   ├── init.py           # Module initialization script
│   └── *.py              # Additional source files
├── wwwroot/              # Frontend scripts and resources (.ui, .dspy files)
├── models/               # Database table definitions as JSON files
├── json/                 # CRUD operation definitions as JSON files  
├── init/                 # Module initialization data
│   └── data.json         # Initial data in specified format
├── scripts/              # Supporting scripts
│   └── load_path.py      # RBAC permission registration script (see RBAC section below)
├── skill/                # MANDATORY — AI agent reference documentation
│   └── SKILL.md          # Agent-facing module spec: data model, endpoints, pitfalls
├── pyproject.toml        # Python packaging configuration
└── README.md             # Module documentation
```

## Core Implementation Requirements

### 1. Module Initialization (init.py)
- **Primary purpose**: Register all module functions with ServerEnv instance so they can be called directly from .ui and .dspy files
- **NOT for route registration**: wwwroot files are automatically routed via `/{module_name}/filename.ext`
- **ServerEnv configuration is REQUIRED for all modules**: Every module must register its functions with ServerEnv in the `load_{modulename}()` function
- **CRITICAL: Export functions in __init__.py**: All async functions defined in `init.py` must be explicitly imported in the package's `__init__.py` file, otherwise Sage framework cannot find them and dspy calls will fail with `NameError: name 'xxx' is not defined`

**Pitfall: Triple-place function registration — adding or removing functions requires updating THREE files**

When adding or removing a module function, THREE files must be updated in sync:
1. **`mymodule/mymodule.py`** (or wherever the function is defined) — the function implementation
2. **`mymodule/__init__.py`** — the import/export line
3. **`mymodule/init.py`** — the `env.xxx = xxx` registration in `load_{module}()`

Forgetting any one causes runtime errors:
- Missing #2 → `ImportError` or `AttributeError` when init.py tries to import
- Missing #3 → `NameError: name 'xxx' is not defined` in .dspy/.ui templates
- Removing from #1 but not #2/#3 → `ImportError` at module load time

**Cleanup checklist when removing a function:**
```
grep -rn 'function_name' mymodule/ --include='*.py'
# Must appear in exactly: definition, __init__.py import, init.py env registration
# Remove from ALL three, then commit.
```

Example `__init__.py` (required):
```python
# mymodule/__init__.py
from .init import (
    create_user,
    get_user_list,
    update_user_profile,
    # ... export all public API functions
)
```

- Keep init.py focused on ServerEnv registration and minimal initialization
- Include a `load_{modulename}()` function that registers all public functions with ServerEnv
- Functions registered with ServerEnv become directly callable from .ui templates and .dspy scripts

Example structure:
```python
from ahserver.serverenv import ServerEnv

# Module metadata and constants  
MODULE_NAME = "mymodule"
MODULE_VERSION = "1.0.0"

# Helper functions for web scripts
def get_data():
    """Get data - will be registered with ServerEnv"""
    pass

def save_data(data):
    """Save data - will be registered with ServerEnv"""  
    pass

def load_mymodule():
    """Register all functions with ServerEnv so they can be called from .ui/.dspy files"""
    env = ServerEnv()
    env.get_data = get_data
    env.save_data = save_data
    return True
```

**Pitfall: CRUD dspy wrappers use plural table names — register both singular and plural**

The `xls2ui` tool generates CRUD wrapper `.dspy` files (e.g., `add_suppliers.dspy`) that reference functions with **plural** names matching the table name: `create_suppliers`, `update_suppliers`, `delete_suppliers`. But `init.py` typically defines functions with **singular** names (e.g., `create_supplier`). If only the singular name is registered, the CRUD UI will get `NoneType` errors.

**Fix**: Register both forms in `load_{module}()`:
```python
env.create_supplier = create_supplier
env.create_suppliers = create_supplier  # xls2ui uses plural
env.update_supplier = update_supplier
env.update_suppliers = update_supplier
env.delete_supplier = delete_supplier
env.delete_suppliers = delete_supplier
```

**Symptom**: CRUD "add" button returns 500 with `function not found` — the wrapper dspy calls `create_suppliers` but only `create_supplier` was registered.

**Pitfall: Subagent Delegation for Module Creation — ALWAYS Validate After**

When using `delegate_task` to create modules in parallel, subagents **consistently** produce errors even when given correct spec instructions (observed 2/3 failure rate):

1. **File misplacement**: `init.py` and `__init__.py` placed at module root (`mymodule/init.py`) instead of package dir (`mymodule/mymodule/init.py`)
2. **Wrong model JSON format**: Uses `{"table":"...", "fields":{...}}` instead of `{"summary":[...], "fields":[...], "indexes":[...], "codes":[...]}`
3. **Wrong CRUD JSON format**: Uses `{"table":"...", "list":{...}}` instead of `{"tblname":"...", "params":{"browserfields":{...}, "editable":{...}}}`
4. **Hallucinated sqlor APIs**: Invents `sqlor.save/list/one/delete/insert/query` — these methods DO NOT EXIST. Only `sor.C/U/D/R/I/sqlExe` are valid.

**Mandatory post-delegation validation** (see `references/bulk-module-creation-pattern.md` "Subagent Validation Checklist"):
- Move misplaced init.py/__init__.py to package dir
- Validate all model JSONs have `summary` key with array primary
- Validate all CRUD JSONs have `tblname` and `params.editable`
- Grep for hallucinated sqlor APIs and fix to sor.C/U/D pattern
- Run dspy audit (no imports, no print, no uuid)
- Run py_compile on all .py files

**Pitfall: debug() in dspy does NOT include filename — always prefix manually**

`debug()` calls in `.dspy` files log as `[sage][debug][<string>:N]` — the framework injects dspy code into a `<string>` context, so the file path is never shown. When tracing production issues across multiple dspy files, all debug output looks identical.

**Fix**: Always prefix debug calls with the file name:

```python
# WRONG — no way to tell which dspy this came from
debug(f'{params_kw=}')

# CORRECT — immediately identifiable in logs
debug(f'product_category_create.dspy: START params_kw={dict(params_kw)}')
debug(f'get_product_category.dspy: ns keys={list(ns.keys())}')
```

This is essential when debugging multi-step flows (e.g. add → refresh) where several dspy files execute in sequence. Each debug line becomes self-identifying.

**Pitfall: DSPY API handlers must forward ALL client params — never hardcode dispatch fields**

When a `.dspy` API endpoint acts as a task dispatcher (submitting to a longtasks worker or downstream service), it MUST forward the client's parameters. Hardcoding dispatch fields like `task_type` silently ignores client intent:

```python
# WRONG — hardcodes task_type, ignores client's task_type
payload = {'task_type': 'separate', 'task_id': task_id, 'audio_path': audio_path}
await longtasks.submit_task(payload)

# CORRECT — reads task_type from request, forwards all params
task_type = params_kw.get('task_type', 'separate')
payload = {'task_type': task_type, 'task_id': task_id, 'audio_path': audio_path}
if params_kw.get('output_dir'):
    payload['output_dir'] = params_kw['output_dir']
await longtasks.submit_task(payload)
```

**Symptom**: Client sends `task_type: "separate_full"` but worker log shows `task_type: "separate"`. Worker always runs the default mode regardless of what client requested. Takes hours to trace because the worker code is correct — the DSPY is silently dropping the parameter.

**Audit**: grep API DSPY files for hardcoded `'task_type'`, `'mode'`, or similar dispatch strings inside payload construction.

**Pitfall: py_compile is INVALID for .dspy files**

`.dspy` files are injected into an async function context at runtime — `return` at top level is correct. Running `py_compile` on `.dspy` files produces false positives (`'return' outside function`, `'await' outside function`). Only use `py_compile` for `.py` files (like `init.py`). For `.dspy` validation, use the dspy audit script (grep-based checks).

**CRITICAL: dspy MUST use explicit `return` — ahserver wraps code in `async def`.** The dspy handler prepends `async def myfunc(request, **ns):` and awaits the function. Setting `result` as a bare expression without `return` causes the function to return `None`. See `references/dspy-execution-and-module-structure.md` for the full execution model, pre-loaded globals, Tree widget data format, and module symlink patterns.

**Pitfall: Sage startup requires environment variables — cannot test locally without them**

Sage's `app/sage.py` imports modules that read env vars at import time (e.g., `ALIPAY_PUB` for payment keys). Starting Sage locally without these vars causes `FileNotFoundError`. Use syntax checks (`py_compile` for .py files) and dspy audit scripts instead of full startup for local validation.

**Pitfall: Raw SQL column names — ALWAYS verify against model.json, never guess**

When writing raw SQL (`sor.sqlExe`) or ORM calls that reference table columns, ALWAYS verify the column names against the actual model definition (`models/{table}.json`), not from memory or similar tables. Guessing column names (e.g., `permcode` instead of `path`, `permname` instead of `name`) causes hard-to-debug runtime errors that require multiple fix attempts.

**Check before writing SQL:**
```bash
python3 -c "import json; cols=[f['name'] for f in json.load(open('models/table.json'))['fields']]; print(cols)"
```

**Common wrong guesses → correct names in Sage:**
| Table | Wrong guess | Correct (from model) |
|-------|-----------|---------------------|
| permission | permcode, permname | path, name |
| users | created_date | created_at |
| organization | org_name | orgname |



**Pitfall: `{{id}}` in uapi response templates resolves to Python built-in `id()`**

In Sage uapi response templates, Jinja2 resolves `{{id}}` as Python's `id()` built-in function, not the upstream API's `id` field. This produces `<built-in function id>` in the response instead of the actual task/request ID.

**Fix**: Never use `{{id}}` in uapi response templates. Always map to a different field name:
```python
# WRONG — {{id}} resolves to Python id() function
'{"taskid":"{{id}}","status":"{{status}}"}'

# CORRECT — map via request context or use a different variable
'{"taskid":"{{taskid}}","status":"{{status}}"}'
```

**Affected**: Any uapi where the upstream response has a field called `id`. Sage's Jinja2 context silently shadows it with the Python built-in.

**Pitfall: uapiio `input_fields` MUST be a JSON array for bricks.js compatibility**

The bricks.js `LlmIO` widget calls `this.input_fields.forEach(...)`. If `input_fields` is a JSON object `{"field": {...}}` instead of a JSON array `[{"name":"field", ...}]`, the browser throws:
```
TypeError: this.input_fields.forEach is not a function
```

**Fix**: Always format uapiio `input_fields` as an array of objects:
```json
// WRONG — object format, bricks.js can't iterate
{"prompt":{"type":"string","required":true}}

// CORRECT — array format with name/label/uitype
[{"name":"prompt","label":"用户输入","uitype":"text","required":true}]
```

**Reference**: Use existing KTV uapiio records (e.g., `ktv_asr_transcribe_io`) as the canonical format reference.

**Pitfall: `json.dumps()` in uapi data templates causes `data=None`**

When a uapi data template uses `{{json.dumps(prompt, ensure_ascii=False)}}`, the Jinja2 rendering can fail silently, resulting in the HTTP request body being `None`. This logs as `body=None` in the uapi debug output.

**Fix**: Use plain string interpolation `"{{prompt}}"` instead of `json.dumps()`. The model name should be hardcoded to the upstream service ID (e.g., `"model":"hy-image-v3.0"`), not passed through `{{model}}`.

```python
# WRONG — json.dumps can render to None
'{"model":"{{model}}","prompt":{{json.dumps(prompt,ensure_ascii=False)}}}'

# CORRECT — plain string, hardcoded service model ID
'{"model":"hy-image-v3.0","prompt":"{{prompt}}"}'
```

**Model ID sourcing**: The model ID sent to the upstream must be the provider's actual service ID. Discover actual IDs by querying the provider's `/v3/models` endpoint (ARK) or `/v1/models` endpoint (tokenhub), then hardcode them in the uapi data template.

**Pitfall: `return` inside `async with` → silent NoneType in .dspy files**

The ahserver framework wraps .dspy code in `async def myfunc(request, **ns):` and awaits it. Returning from inside an `async with db.sqlorContext(...) as sor:` block causes the function to return `None`, producing `return data type error, <class 'NoneType'>`.

**Fix**: Collect all results inside the `async with` block, then `return` AFTER the block exits:

```python
# WRONG — return inside async with → NoneType
async with db.sqlorContext(dbname) as sor:
    recs = await sor.R('table')
    return json.dumps({'total': len(recs), 'rows': rows})

# CORRECT — collect inside, return outside
rows = []
async with db.sqlorContext(dbname) as sor:
    recs = await sor.R('table')
    for r in (recs or []):
        rows.append({'id': r.id, 'name': r.name})
return {'total': len(rows), 'rows': rows}
```

**Symptom**: `Exception: /path/file.dspy return data type error, <class 'NoneType'>` with no other traceback. Minimal test: `return {'ok': True}` inside async with also fails.

**Pitfall: `sor.C()` silently drops records when `created_at` is missing**

sqlor's `sor.C()` does NOT auto-set timestamp fields. If the model defines `created_at TIMESTAMP NOT NULL`, `sor.C` may succeed silently but the record never reaches the database. Always include `'created_at': curDateString()`:

```python
ns = dict(params_kw)
ns['id'] = params_kw.get('id', getID())
ns['created_at'] = curDateString()   # REQUIRED — sor.C does not auto-set
ns['org_id'] = (await get_userorgid()) or '0'
async with db.sqlorContext(dbname) as sor:
    await sor.C('table_name', ns)
```

**Symptom**: dspy returns `{"success": true}` but `SELECT` returns zero rows.

**Pitfall: Bash heredoc `\$` escapes when writing dspy files via terminal**

When using `cat > file.dspy << 'EOF'` in bash, backslash sequences like `\${pid}\$` intended as sqlor placeholders get written literally as `\${pid}\$` instead of `${pid}$`. This causes sqlor to fail silently because the placeholder pattern is unrecognized.

**Fix**: Never escape `$` in heredocs for dspy files. Write `${pid}$` as-is:

```bash
# WRONG — heredoc writes literal backslash
cat > file.dspy << 'EOF'
sql = 'WHERE id=\${pid}\$'
EOF

# CORRECT — no escaping needed in single-quoted heredoc
cat > file.dspy << 'EOF'
sql = 'WHERE id=${pid}$'
EOF
```

**Verification**: `grep 'project_id' file.dspy | cat -A` — look for `\$` (shows as `\$`) which means the backslash was written literally. Correct output shows `${pid}$` with no backslash.

**Pitfall: Do NOT name module functions `get_module_dbname`**

In `init.py`, do NOT define a function named `get_module_dbname()` — this name is already provided as a global by ServerEnv and is used by .dspy scripts. Registering it via `env.get_module_dbname = ...` will overwrite the global. If you need a module-specific dbname helper, name it with a private prefix like `_get_dbname()`.

**Pitfall: Do NOT name module functions `get_module_dbname`**

**Key Points:**
- All functions that need to be accessible from .ui/.dspy files must be registered with ServerEnv
- The `load_{modulename}()` function is called by Sage during module loading
- After registration, functions can be called directly in .ui templates: `{% set data = get_data() %}`
- This pattern is used by all standard modules (rbac, appbase, etc.)

### 2. Frontend Development
- **Mandatory**: Use bricks-framework for all frontend interfaces
- Store all .ui files as **pure JSON format** (NOT HTML/CSS) in wwwroot/
- See `references/bricks-ui-pitfalls.md` for id placement (widget level, not options), Button click event, Popup/Form patterns, and script actiontype Jinja2 limitations.
- **Automatic routing**: Files in wwwroot/ are automatically accessible via `/{module_name}/filename.ext`
- **No manual route registration needed**: The framework automatically serves .ui and .dspy files
- .ui files contain component definitions with widgettype, options, subwidgets, and binds properties
- Store CSS files in wwwroot/ for styling extensions
- Store JavaScript files in wwwroot/ for custom functionality and registerFunction extensions
- **ahserver auto-serves .css and .js files**: files placed in wwwroot/ are automatically discovered and injected by ahserver into the HTML response — **DO NOT** manually add `<link>` or `<script>` tags in .ui files to reference them. This is a common mistake that causes unnecessary 403 errors when RBAC is enabled.
- **WARNING**: Even though ahserver auto-serves .css/.js files, they are **STILL subject to RBAC checks**. Every .css and .js file must be registered in `load_path.py`. Files needed before authentication (e.g., theme CSS) should use `any` role.
- Store .dspy files (controlled Python scripts) in wwwroot/ for backend logic
- The build.sh script will link module wwwroot files to the main application wwwroot directory

#### 2.0 Module Entry Point (index.ui) — MANDATORY
- **Every business module MUST have `wwwroot/index.ui`** as its user-facing entry point
- Exception: reference/foundation modules (rbac, appbase, accounting, apppublic, sqlor, ahserver) are exempt
- index.ui should integrate ALL the module's .ui and .dspy files into a single navigation page
- Use `ResponsableBox` with clickable card widgets (VBox + binds click → urlwidget) to link to each feature page
- Use `{{entire_url('filename.ui')}}` or `{{entire_url('api/xxx.dspy')}}` for ALL url values — this converts server-relative paths to the correct runtime URL prefix
- Default content area: a VBox with `id: "app.<module>_content"` for urlwidget targets to load pages into

Example index.ui pattern:
```json
{
    "widgettype": "VBox",
    "options": {"width": "100%", "height": "100%", "padding": "20px"},
    "subwidgets": [
        {"widgettype": "Text", "options": {"label": "模块名称", "fontSize": "24px"}},
        {"widgettype": "ResponsableBox", "options": {"gap": "16px", "minWidth": "250px"},
         "subwidgets": [
            {"widgettype": "VBox",
             "options": {"backgroundColor": "#FFFFFF", "padding": "20px", "cursor": "pointer"},
             "binds": [{
                 "wid": "self", "event": "click", "actiontype": "urlwidget",
                 "target": "app.module_content",
                 "options": {"url": "{{entire_url('feature_page.ui')}}"},
                 "mode": "replace"
             }],
             "subwidgets": [{"widgettype": "Text", "options": {"label": "功能名称"}}]}
         ]},
        {"widgettype": "VBox", "id": "module_content",
         "options": {"width": "100%", "flex": "1", "marginTop": "20px"}}
    ]
}
```

#### 2.1 URL Path Rules — MANDATORY
- **ALL** url fields in .ui files MUST use `{{entire_url(url)}}` format
- This applies to: Menu items, Button url, Form url, Iframe url, urlwidget targets, and any other url property
- `{{entire_url()}}` converts server-relative paths (e.g., `feature.ui`, `/module/page.ui`) to the correct runtime URL including any prefix configured in `config.json`
- **WRONG**: `"url": "/module/page.ui"` or `"url": "feature.ui"`
- **CORRECT**: `"url": "{{entire_url('/module/page.ui')}}`" or `"url": "{{entire_url('feature.ui')}}`"
- Do NOT use `{{entire_url()}}` for: `data:` URIs, `#` anchors, `http(s)://` URLs, or variable references `${var}`

### 2.2 JSON CRUD File URL Rules — MANDATORY
- `json/` 目录下的 CRUD 定义文件（如 `json/handover_list.json`）使用 `entire_url()` 引用同模块的其他 CRUD 别名时，**必须使用 `../` 相对路径**
- 原因：JSON 文件位于 `json/` 子目录中，而 `.dspy` CRUD 文件和生成的 `.ui` 文件位于模块根目录（即 `wwwroot/`），需要跳出 `json/` 目录
- 适用于：`subtables[].url`、`browserfields` 中的 `dataurl` 等所有 `entire_url()` 引用
- **WRONG**: `"url": "{{entire_url('handover_items_list')}}"`\n- **CORRECT**: `"url": "{{entire_url('../handover_items_list')}}"`
- 引用其他模块的别名（如 `appcodes_list`）不需要 `../`，因为通过模块间路由解析

### 2.2a CRUD JSON `new_data_url` — Do NOT Pass `editexclouded` Fields as Query Params

**Pitfall**: When `new_data_url` passes a field as a query param (`?supplier_id=...`) that is also in `editexclouded`, the Form POST body and URL query params get merged into `params_kw` as a **list** (e.g., `supplier_id: ['val', 'val']`). `sor.C` receives a list instead of a string, MySQL interprets it as a subquery, and throws `Operand should contain 1 column(s)`.

**Fix**: remove the query param from `new_data_url`. If the field is excluded from the form (`editexclouded`), don't pass it in the URL. If it needs to be pre-set for subtable context, add it as a hidden field in the form instead.

```json
// WRONG
"new_data_url": "{{entire_url('../api/create.dspy')}}?supplier_id={{params_kw.get('supplier_id','')}}"

// CORRECT
"new_data_url": "{{entire_url('../api/create.dspy')}}"
```
- 适用于：`subtables[].url`、`browserfields` 中的 `dataurl` 等所有 `entire_url()` 引用
- **WRONG**: `"url": "{{entire_url('handover_items_list')}}"`
- **CORRECT**: `"url": "{{entire_url('../handover_items_list')}}"`
- 引用其他模块的别名（如 `appcodes_list`）不需要 `../`，因为通过模块间路由解析

### 2.3 JSON CRUD File editable Section — MANDATORY
- **所有用于 DataViewer/Tabular 列表展示的 JSON 配置文件必须包含 `editable` 段**，否则"新增/编辑/删除"按钮无法提交数据
- `editable` 段定义表单提交的三个目标 URL，指向 `wwwroot/api/` 下的 `.dspy` 脚本
- **重要**：`wwwroot` 是 Web 根目录，在 URL 层不可见，路径中**不能**出现 `wwwroot/`
- 模板中的 CRUD 别名在 `json/` 目录下，`.dspy` 文件实际在 `wwwroot/api/`，但 URL 路径用 `../api/`（跳过 wwwroot）
- **WRONG**: JSON 中没有 `new_data_url`/`update_data_url`/`delete_data_url`，或者路径包含 `wwwroot`，或者这些字段嵌套在 `editable` 对象中
- **CORRECT** — `new_data_url`/`update_data_url`/`delete_data_url` **必须放在 `params` 顶层**，不能嵌套在 `editable` 对象内：
```json
{
    "tblname": "tablename",
    "alias": "alias_name",
    "title": "模块名称",
    "params": {
        "browserfields": { ... },
        "new_data_url": "{{entire_url('/module/api/tablename_create.dspy')}}",
        "update_data_url": "{{entire_url('/module/api/tablename_update.dspy')}}",
        "delete_data_url": "{{entire_url('/module/api/tablename_delete.dspy')}}"
    }
}
```

**Pitfall: `editable` nesting causes `new_data_url` to be silently ignored.** The xls2ddl template reads `new_data_url` from `params` top level (`{% if new_data_url %}`), NOT from `params.editable.new_data_url`. Nesting these fields inside an `editable` object causes xls2ui to generate the default `add_{tablename}.dspy` instead — the CRUD falls back to framework default insert, and any custom create logic is silently bypassed.

**Real case (2026-07-13):** discount module had `"editable": {"new_data_url": "...", ...}` nested. xls2ui generated `add_discount_marketing.dspy` instead of using `marketing_create.dspy`. `created_by` field was never set by custom handler. Fixed by moving to params top level and using `/module/api/` absolute paths.
- `_edit.json` 文件（表单编辑页配置）同样需要 `editable` 段
- 三个 `.dspy` 文件必须存在于 `wwwroot/api/` 目录：`xxx_create.dspy`、`xxx_update.dspy`、`xxx_delete.dspy`
- 如果缺少 `editable` 段，DataViewer 的 `build_add_form()` / `build_update_form()` 会因为 `this.editable.new_data_url` 为 undefined 而提交失败
- 完整 JSON 配置模板见 `references/json-config-pattern.md`
- 自定义工具栏按钮 + PopupWindow bind 模式见 `references/crud-json-toolbar-bind-pattern.md`

#### 2.3b bricks Toolbar Bind — Only 5 Valid Actiontypes

**Pitfall**: bricks.js bind 的 `actiontype` 只有 5 种有效值：`urlwidget`、`script`、`url`、`datawidget`、`event`。`dspy` 和 `function` 不是有效 actiontype。工具栏调用 dspy 端点的正确模式：

```json
{
    "actiontype": "urlwidget",
    "target": "PopupWindow",
    "popup_options": {"title": "标题", "height": "200px", "width": "400px"},
    "params_mapping": {"mapping": {"id": "target_param"}, "need_other": false},
    "options": {"method": "POST", "url": "{{entire_url('../api/xxx.dspy')}}"}
}
```

dspy 返回值被 bricks 渲染在 PopupWindow 中。如果 dspy 返回 `DataViewer` widget 嵌入 CRUD 页，url 必须带 `_webbricks_=1` 参数。

**禁止**：`script` actiontype 中使用 `fetch()` / `bricks_fetch()` — bricks 原则明确禁止 script 中的 fetch/setInterval。

#### 2.3c Module-Internal entire_url() Must Use Absolute /module/ Paths

**Pitfall**: 模块内 .ui 文件（index.ui、menu.ui）中 `entire_url('supply_contracts_list')` 可能解析为 `/supply_contracts_list`（缺模块前缀 `/supplychain/`），RBAC 检查用缺前缀路径 → 403。

**Fix**: 所有模块内部的 URL 使用绝对路径：

```jinja2
{# WRONG: 缺模块前缀 → 403 #}
{{entire_url('supply_contracts_list')}}

{# CORRECT #}
{{entire_url('/supplychain/supply_contracts_list')}}
```

**检查方法**：
```bash
grep "entire_url(" wwwroot/index.ui wwwroot/menu.ui | grep -v '/supplychain/' && echo "MISSING PREFIX"
```

#### 2.3d dspy Returned DataViewer URL Must Be Absolute

**Pitfall**: dspy 返回的 `DataViewer` widget 中相对路径 `../distribution_agreements_list` 在 PopupWindow 上下文中解析错误，缺模块前缀。

**Fix**: dspy 中用绝对路径（`entire_url()` 是 Jinja2 函数，dspy 中不可用）：

```python
return {
    'widgettype': 'DataViewer',
    'options': {
        'url': f'/supplychain/supply_contracts_list?supplier_id={supplier_id}&_webbricks_=1'
    }
}
```

### 2.4 Menu Widget — Do NOT use binds
- **Menu控件已经内部处理了click事件**，不需要额外添加 binds 绑定
- Menu items 的 url 字段会自动处理导航，只需设置 `url: "{{entire_url('page.ui')}}`" 即可
- **WRONG**: 在 Menu 上添加 binds 处理 itemclick
- **CORRECT**: 在 Menu items 中设置 url，Menu 自动处理点击

#### 2.1 DSPY File Implementation Guidelines
- **Purpose**: .dspy files provide backend API endpoints that can be called from .ui files
- **HTTP Method Handling**: ahserver supports GET and POST; the framework distinguishes automatically
- **Parameter Extraction**: ALL parameters (query string AND POST body) are automatically parsed by ahserver into `params_kw` — a dictionary-like object. Access via `params_kw.get('key')` or `params_kw.key`. Nested JSON objects/arrays in POST bodies are preserved as dict/list structures in `params_kw`
- **POST Data Reading**: Do NOT use `os.environ`, `os.read()`, or `sys.stdin`. ahserver reads and parses POST data automatically
- **Error Handling**: Always return consistent JSON response structure with `status`, `data`, and optional `message`/`total` fields
- **Security**: Validate all input parameters and implement proper error handling
- **Integration**: Call functions from your module's `init.py` (registered with ServerEnv via `load_{modulename}()`) to perform business logic
- **Debug output**: Use `debug_params('name', dict_obj)` for compact logging instead of `debug(f'{dict_obj=}')` which dumps entire objects. See `references/web-app-and-dspy-spec.md` for full debug output best practices.
- **Extending .dspy globals**: To make a new function available in all .dspy files, inject it via `self.y_env` in `ahserver/processorResource.py`. See `references/web-app-and-dspy-spec.md` section "Injecting New Functions into .dspy Execution Environment".

#### CRITICAL: ServerEnv() is NOT needed in .dspy scripts
- **DO NOT** write `env = ServerEnv()` or `env.get_module_dbname('xxx')` in .dspy files
- All ServerEnv-registered functions are directly available via `globals()`: `get_module_dbname()`, `get_user()`, `get_userorgid()`, `DBPools()`, `password_encode()`, `getID()`, module-specific functions like `create_user_apikey`, etc.
- **WRONG**: `env = ServerEnv(); dbname = env.get_module_dbname('dapi'); create_apikey_func = getattr(env, 'create_user_apikey', None)`
- **CORRECT**: `dbname = get_module_dbname('dapi'); create_apikey_func = create_user_apikey`
- **CORRECT for org_id**: `org_id = (await get_userorgid()) or '0'`
- Only use `ServerEnv()` in Python module files (init.py, *.py), never in .dspy scripts

**Pitfall: `ServerEnv()` is a process-level singleton.** Every call to `ServerEnv()` within a process returns the SAME instance. So `env = ServerEnv()` in `load_*()` and `env = ServerEnv()` deep in `accounting.py` share state. `env.get_module_dbname = ...` set in `backend_accounting.py` is visible everywhere. Do NOT assume each call creates a new instance.

**Pitfall: `ServerEnv()` ≠ `request._run_ns` — NEVER use `ServerEnv()` to get per-request user/session context.** Per-request functions (`get_user()`, `get_userid()`, `get_userorgid()`) are registered on `request._run_ns` (which is `self.y_env`, a plain `DictObject` set up in `processorResource.py:347-349`), NOT on the `ServerEnv()` singleton. Using `env = ServerEnv(); await env.get_user()` silently returns None.

**Wrong (user fields silently become NULL):**
```python
async def create_record(request, params_kw):
    env = ServerEnv()                          # WRONG — singleton, no per-request user
    dbname = env.get_module_dbname('module')   # works (set at startup)
    user_id = await env.get_user()             # SILENTLY RETURNS None
    org_id = await env.get_userorgid()         # SILENTLY RETURNS None
    await sor.C('table', {'created_by': user_id})  # created_by = None! Bug.
```

**Correct:**
```python
async def create_record(request, params_kw):
    env = request._run_ns                      # CORRECT — per-request namespace
    dbname = env.get_module_dbname('module')   # also works (via RF mechanism)
    user_id = await env.get_user()             # Returns actual user ID
    org_id = await env.get_userorgid()         # Returns actual org ID
```

**When IS `ServerEnv()` correct?**
- In `load_{module}()` — runs once at startup, registers global functions
- For startup-time globals when no `request` is available

**Detection command:**
```bash
grep -n 'env = ServerEnv()' module/init.py | while read line; do
  lineno=$(echo "$line" | cut -d: -f1)
  sed -n "$((lineno+1)),$((lineno+6))p" module/init.py | grep -q 'get_user\|get_userorgid\|get_userid' \
    && echo "LINE $lineno: BUG — ServerEnv() has no per-request user context"
done
```

**Real case (2026-07-13):** discount module `create_marketing` used `env = ServerEnv()` → `created_by` always NULL. 7 functions affected. Same root cause as `generate_promo_code` which correctly used `request._run_ns`.

**Pitfall: Module pkgs go under SAGE, not cross-applications.** A module's wwwroot symlink must point to `../pkgs/<module>/wwwroot` under the HOST application's directory. Do NOT place modules under another app's pkgs (e.g., `/pipeline-app/pkgs/tenant/`) — this breaks the module's independence. Each application clones installs modules into its own `pkgs/` directory.

**Example**: `tenant` module should be at `/d/apitest/sage/pkgs/tenant/` with symlink `sage/wwwroot/tenant -> ../pkgs/tenant/wwwroot`. NOT at `/d/apitest/pipeline-app/pkgs/tenant/`.

**Pitfall: sqlor method argument counts differ — `sor.I` is special**

| Method | Args | Pattern |
|--------|------|---------|
| `sor.C` | 2 | `sor.C('table', ns)` |
| `sor.R` | 2 | `sor.R('table', ns)` |
| `sor.U` | 2 | `sor.U('table', ns)` |
| `sor.D` | 2 | `sor.D('table', ns)` |
| `sor.I` | 1 | `sor.I(ns)` |

`sor.I(data)` only takes the data dict — no table name. Passing `sor.I('table', data)` causes `TypeError: I() takes 2 positional arguments but 3 were given`. The table name is resolved from the sqlor context.

## Read-Only / Dashboard Modules

Not all modules need database tables or CRUD operations. For read-only modules (dashboards, statistics, reporting), `models/` and `json/` directories can be omitted entirely.

**Preferred pattern**: `load_XX.py` exposes async data functions via ServerEnv → individual `.ui` files use Jinja2 templates (`{{get_data(request)}}`) → `RefreshWidget` wraps each card for auto-refresh. No `.dspy` files or JS polling needed.

See `references/read-only-module-pattern.md` for:
- Jinja2 .ui + RefreshWidget + load_XX architecture (preferred)
- Legacy .dspy + JS polling pattern (deprecated, still documented)
- ECharts ChartBar/ChartLine integration with inline data or data_url
- Cross-table queries using the shared 'sage' database
- Concurrent user detection pattern

## Database Integration
- **Mandatory**: Use sqlor-database-module for all relational database operations
- Store table definitions as `{tablename}.json` in models/ directory
- Create CRUD definition files as `{tablename_or_alias}.json` in json/ directory
- Follow sqlor database operation specifications
- **Bulk xlsx→json conversion**: use `scripts/xlsx2json_models.py` — converts all `.xlsx` model files across Sage modules to JSON format in one pass

### 4. Initialization Data
- Store in `init/data.json` or `init/data.yaml`. Two formats are supported:

**Format A — Direct table seeding** (arbitrary tables, JSON):
```json
{
    "table1": [
        {"field1": "value1", ...},
        {"field1": "valuen", ...}
    ]
}
```

**Format B — Appcodes registration** (code value groups for appcodes_kv, JSON):
```json
{
    "appcodes": [
        {
            "parentid": "sc_relation_type",
            "parentname": "供销关系合作类型",
            "items": [
                {"k": "distribution", "v": "分销"},
                {"k": "agency", "v": "代理"}
            ]
        }
    ]
}
```

**Format C — Appcodes registration (YAML)** — used by accounting and some other modules:
```yaml
appcodes:
- id: credit_status
  name: 信用额度状态
  hierarchy_flg：0

appcodes_kv:
- id: credit_status_active
  parentid: credit_status
  k: active
  v: 生效
- id: credit_status_inactive
  parentid: credit_status
  k: inactive
  v: 停用
```

**YAML format differences from JSON Format B:**
- `appcodes` entries use `id` (primary key), `name`, `hierarchy_flg` (note: some files use fullwidth colon `：` for `hierarchy_flg` — this is a known pre-existing quirk)
- `appcodes_kv` entries use `id`, `parentid`, `k`, `v` — each entry is a flat record (not nested under `items`)
- The `parentid` in `appcodes_kv` must match an `id` from the `appcodes` section
- When adding new code groups, add BOTH the `appcodes` parent record AND all `appcodes_kv` child records

**Pitfall: YAML lint warnings about `hierarchy_flg：0`** — Some existing data.yaml files use a fullwidth colon (`：`) instead of ASCII colon (`:`) for `hierarchy_flg`. This is a pre-existing issue in the file, not something introduced by new edits. Do not "fix" it unless the user asks — it may break the loader that expects the original format.

### 5. Encoding Management
- Store encodings in appbase module's appcodes and appcodes_kv tables
- appcodes table structure:
  - id: str(32), primary key (can be field name)
  - name: encoding name  
  - hierarchy_flg: str(1), '0'=single-level, '1'=multi-level
- appcodes_kv table structure:
  - id: str(32), primary key
  - parentid: str(32), references parent encoding
  - k: str(32), encoding value
  - v: str(255), display text

**Pitfall: init/data.json parentid too long → appcodes_kv.id exceeds VARCHAR(32)**

The init data loader generates `appcodes_kv.id` as `{parentid}_{k}`. Both `id` and `parentid` columns are `VARCHAR(32)`. If `parentid` is too long, the derived ID overflows. Example: `supplier_settlement_cycle` (27 chars) + `_` + `quarterly` (9 chars) = 37 chars → `DataError: Data too long for column 'id'`.

**Rule**: `len(parentid) + 1 + len(longest_k)` ≤ 32. Parentid must be ≤ 22 chars when k values are typical code words. Keep parentids short (≤ 22 chars). When already referenced in `models/*.json` codes `cond`, update both `init/data.json` and all `models/*.json` files referencing the old parentid.

### 5b. Codes Referencing Future Modules

When a table field references another module that does not yet exist (e.g., `productid` → `products` table from a future `product` module), **still add the codes entry** in the model JSON:

```json
"codes": [
    {
        "field": "productid",
        "table": "products",
        "valuefield": "id",
        "textfield": "product_name"
    }
]
```

This is safe — the codes section only affects UI dropdown rendering and is ignored if the referenced table doesn't exist yet. Do NOT defer adding codes entries until the referenced module is built.

## pyproject.toml Configuration

Configure `pyproject.toml` with the following guidelines:

- **Package name**: Must match the module directory name exactly (e.g., `name = "customer_management"`)
- **Dependencies**: Only declare direct code dependencies, NOT framework packages
  - Use `"sqlor"` (NOT `"sqlor-database-module"`)
  - Use `"bricks_for_python"` (NOT `"bricks-framework"`)
  - Do NOT include foundation packages (`ahserver`, `appbase`, `rbac`, `apppublic`) — these are installed by `build.sh` separately and are not on PyPI
  - Do NOT include `ahserver` — it is installed as a core dependency by `build.sh`
- **pip install . must work locally**: After `build.sh` installs all core dependencies, `pip install .` should succeed without trying to fetch non-PyPI packages

Example:
```toml
[build-system]
requires = ["setuptools>=45", "wheel"]
build-backend = "setuptools.build_meta"

[project]
name = "mymodule"
version = "1.0.0"
description = "Module description"
requires-python = ">=3.8"
dependencies = [
    "sqlor",
    "bricks_for_python",
]

[tool.setuptools.packages.find]
where = ["."]
include = ["mymodule*"]
```

## Project Files
- **pyproject.toml**: Python packaging configuration matching module name
- **README.md**: Comprehensive module documentation — must describe the module's purpose, features, data tables, installation, and integration in plain language. Never leave it empty or boilerplate-only.
- **skill/SKILL.md**: MANDATORY agent-facing reference — YAML frontmatter (name, description) + markdown body covering: module architecture, data model, key DSPY endpoints, and pitfalls. This is the file AI agents read to understand how to work with the module. Every independent module repo must have this file.
- **build.sh**: Module build script that should be integrated into the main application's build process

### skill/SKILL.md Content Guidelines

Every module's `skill/SKILL.md` must contain at minimum:

1. **YAML frontmatter**: `name` and `description` fields
2. **Architecture overview**: Key directories and files, what each does
3. **Data model**: Tables + key columns (even for interaction-layer modules, note they have no tables)
4. **Key endpoints**: DSPY files and what they do
5. **Pitfalls**: Module-specific traps — wrong API patterns, column name gotchas, DBPools quirks

Keep it concise. This is consumed by AI agents, not humans reading for the first time.

## Build Process Integration
When the main application's build.sh runs, it will process **all modules** (including foundation modules like appbase, rbac, and business modules like contract_management) that have models/ or json/ directories:

1. Install `xls2ddl` tool in the virtual environment
2. For modules with `models/` directory containing `.xlsx` files: run `xls2ddl mysql . > mysql.ddl.sql`
3. For modules with `models/` directory containing `.json` files: run `json2ddl mysql . > mysql.ddl.sql`  
4. For modules with `json/` directory: run `xls2ui -m ../models -o ../wwwroot ${modulename} *.json` to generate CRUD UI files
5. Create symbolic links from module wwwroot to main application wwwroot

### CRITICAL: Symlink Generated CRUD Directories

The `xls2ui` tool generates **subdirectories** in `wwwroot/` for each CRUD definition (e.g., `wwwroot/product_category_tree/`, `wwwroot/product_list/`). Each directory contains auto-generated `index.ui`, `get_*.dspy`, `add_*.dspy`, `update_*.dspy`, `delete_*.dspy`. These directories are NOT covered by the individual-file symlink rules above and MUST be linked separately:

```bash
# Link generated CRUD directories (NOT individual files):
cd /path/to/sage/wwwroot/module_name/
ln -sf /path/to/module/wwwroot/crud_alias_name/ .

# Or in build.sh, auto-link all non-api/non-assets directories:
for d in "$MODULE_DIR/wwwroot"/*/; do
    [ -d "$d" ] || continue
    dname=$(basename "$d")
    case "$dname" in api|styles|scripts) continue ;; esac
    ln -sf "$d" "$MODULE_WWWROOT/$dname"
done
```

**Symptoms**:
- CRUD pages return 404 even though `xls2ui` ran successfully — the generated directories exist in the module's wwwroot but aren't linked to Sage's wwwroot.
- ahserver raises `processorResource.py: raise Exception(f'{str(request.url)=} invalid path')` — this specific error means the URL path is not registered in ahserver's route table, which almost always means the file/directory is not symlinked to the deployment target's wwwroot.

### CRITICAL: CRUD-Generated Directories Are READ-ONLY — Never Modify Files Inside Them

**Hard rule**: The `xls2ui` tool generates subdirectories in `wwwroot/` (e.g., `wwwroot/product_category_tree/`, `wwwroot/product_list/`). Every file inside these directories — `index.ui`, `get_*.dspy`, `add_*.dspy`, `update_*.dspy`, `delete_*.dspy` — is **auto-generated** and must NEVER be edited directly.

**JSON manipulation safety — use json.load/dump, not string insertion**: Complex `.ui` files (deeply nested JSON) are notoriously error-prone to modify via string insertion. A single missing comma or bracket silently breaks the page. **Always prefer**: read with `json.load()` → modify the Python dict → write with `json.dump()`. The `json` module guarantees valid output; string replacement does not.

**Pitfall: xls2ddl `data_new_tmpl` uses `{{summary[0].pkey}}` → `WHERE None = %s`**
In xls2ddl commit fc91486 (June 2026), the `data_new_tmpl` was updated to add a post-INSERT SELECT to return `user_data` for tree widgets. It used `{{summary[0].pkey}}` but model JSONs use `primary` (an array), NOT `pkey`. Jinja2 resolves the missing attribute to Python `None`, which renders as `${None}$` in the generated SQL. sqlor then produces `WHERE None = %s`, causing `OperationalError: (1054, "Unknown column 'None' in 'WHERE'")` on every add operation across all CRUD tables.

**Fix**: Change `{{summary[0].pkey}}` to `{{summary[0].primary[0]}}` in `xls2ddl/xls2ddl/tmpls.py` line 308.

**To change behavior of CRUD-generated files, you MUST:**
1. Modify the source configuration: `json/{alias}.json` (CRUD definition) or `models/{table}.json` (table definition)
2. Re-run `build.sh` (which calls `xls2ui`) to regenerate the directory
3. The regenerated files overwrite whatever was there

**When json/ config cannot express the needed logic:**
1. Create a custom dspy in `wwwroot/api/` (e.g., `api/add_user.dspy`, `api/update_user.dspy`)
2. Point to it from `json/{alias}.json` via `new_data_url` / `update_data_url` / `delete_data_url`:
   ```json
   "params": {
       "new_data_url": "{{entire_url('/rbac/api/add_user.dspy')}}",
       "update_data_url": "{{entire_url('/rbac/api/update_user.dspy')}}"
   }
   ```
3. Register the api paths in `scripts/load_path.py` (RBAC) and `sage/load_path.py` (routing)
4. Re-run `xls2ui` to regenerate — the generated `index.ui` will use the custom URLs

**Pitfall: bricks Tabular code fields send values as lists — unwrap before sor.C/U**

When a CRUD JSON config uses `alters` with code lookups (e.g., `supplier_id` → suppliers table), the bricks Tabular form may submit the field value as a **list** `['val1', 'val1']` instead of a plain string.

**Root cause and correct fix**: The list comes from `new_data_url` including the field as a query param (`?supplier_id=...`) while the field is also in `editexclouded`. When the form submits, ahserver merges URL params + form body into `params_kw`, creating a list. **The correct fix is the JSON config**: remove the duplicate from `new_data_url` if the field is in `editexclouded`. Do NOT add list-unwrap workarounds in every dspy.

**Common customizations in api/ dspy:**
- Set default values (e.g., `ns['created_at'] = curDateString()`)
- Auto-generate required business codes (e.g., `contract_code`) — see `references/custom-create-dspy-required-fields.md`
- Clean `_text` suffix fields before sor.U/C (Tabular sends these from code lookups)
- Preserve password on update (pop empty password from ns)
- `try/except` with `format_exc()` + `str(e)` to show real errors to frontend

**WRONG**: Editing `wwwroot/product_category_tree/index.ui` or any `.dspy` inside it
**CORRECT**: Editing `json/product_category_tree.json`, then re-running `build.sh`; OR creating `wwwroot/api/custom.dspy` + pointing json config to it

**Symptom**: User will reject and correct you immediately if you modify files in these directories. These are build artifacts, not source code.

**CRITICAL: Model JSON changes require xls2ui rebuild.** Changing `models/*.json` (codes, fields, indexes) does NOT auto-update the generated CRUD `.ui` files. You MUST re-run `build.sh` (which calls `xls2ui`) after ANY model JSON change. Otherwise the filter dropdowns, edit forms, and data views continue using the old model. This is the #1 cause of "I changed the model but the page still shows old data." See `references/model-change-rebuild-pitfalls.md` for full details including `.xlsx`/`.json` load order, `get_code.dspy` module.table resolution failure, and collation mismatch pitfalls.

**Pitfall: model change → rebuild → THREE post-rebuild checks required**

After running `build.sh` for a module, verify THREE things before declaring done:
1. **Files exist**: `ls wwwroot/<module>/<alias>/index.ui` — rebuild can fail silently
2. **Symlinks intact**: if alias-based CRUD URLs use root-level paths (`/<alias>` instead of `/<module>/<alias>`), ensure symlinks survive the rebuild: `ls -la sage/wwwroot/<alias>` → should point to `sage/wwwroot/<module>/<alias>`
3. **RBAC refreshed**: new index.ui has new `dataurl`/`get_code.dspy` calls → verify permissions exist AND restart Sage (RBAC cache has 10-min TTL; `load_path.py` only inserts new permissions, doesn't assign roleperm for existing ones)

**Pitfall: xls2ui strips code-type fields from generated editexclouded**

When a field appears in BOTH `browserfields.alters` (with `uitype: "code"` and `dataurl`) AND `editexclouded` in the source CRUD JSON config, `xls2ui` strips it from the generated `index.ui`'s `editexclouded` list. This causes bricks.js to render a UiCode dropdown widget in the add/edit form, which calls `.build_options()` on undefined data.

**Symptom**: `TypeError: Cannot read properties of undefined (reading 'length')` at `bricks.UiCode.build_options` when clicking Add/Edit. Stack trace: `bricks.Tabular.build_add_form` → `bricks.InlineForm` → `new bricks.UiCode` → `.build_options()`.

**Diagnosis**: Compare the source CRUD JSON `params.editexclouded` against the generated `wwwroot/{alias}/index.ui`'s `row_options.editexclouded`. Code fields with `dataurl` in `alters` will be missing.

**Fix**: Add the missing field(s) back to `editexclouded` in the generated `index.ui`. Since these files are `.gitignore`'d, use `git add -f`. Note: the fix is undone if `xls2ui` re-runs (via `build.sh`), so deploy with `git pull` only, skipping `build.sh`. Long-term: fix `xls2ui` to not remove fields from `editexclouded` just because they appear in `alters`.

**Verification script** (adapt for the module):
```bash
for alias in supply_contracts_list distribution_agreements_list; do
  want=$(python3 -c "import json; print(set(json.load(open('json/${alias}.json'))['params']['editexclouded']))")
  got=$(python3 -c "
import json,re
t=open('wwwroot/${alias}/index.ui').read()
m=re.search(r'\"editexclouded\"\s*:\s*\[(.*?)\]',t,re.DOTALL)
print(set(s.strip().strip('\"') for s in m.group(1).split(',') if s.strip()))
")
  python3 -c "missing=${want} - ${got}; assert not missing, f'${alias}: {missing}'"
done && echo "OK"
```

**Pitfall: cheight on CRUD index.ui causes title row to fill viewport**

Generated `wwwroot/{alias}/index.ui` files contain `"cheight":40` on both the outer VBox and inner Tabular options. This hardcodes a character-height value that causes the title/header row to expand and fill the entire screen, obscuring the data table.

**Symptom**: CRUD page title bar takes up full viewport height; data table invisible or pushed far below.

**Fix**: Remove ALL `cheight` properties from the generated `index.ui`. Tabular handles its own height automatically:
```diff
- "options":{"cheight":40,"width":"100%"},
+ "options":{"width":"100%"},
```
And inside the Tabular options:
```diff
  "options":{
      "width":"100%",
-     "cheight":40,
```

**Affected files**: Every `wwwroot/{alias}/index.ui` generated by xls2ui. Check all aliases:
```bash
grep -rn 'cheight' wwwroot/*/index.ui
```

### CRITICAL: Do NOT Replace Framework-Generated Code with Custom Scripts

**Core principle**: The CRUD framework and its auto-generated endpoints are **base functionality** — stable, tested, and maintained. Do NOT replace them with hand-written scripts unless there is a genuine special requirement that the framework cannot handle.

**Wrong approach**: Creating `wwwroot/api/llm_list.dspy` to replace the auto-generated `get_llm.dspy` just to add `_text` fields for display.

**Correct approach**: Use the framework auto-generated endpoints, and solve display issues through proper configuration (e.g., fix the `dataurl` API to return `[{field_name, field_name_text}]` format).

**Why this matters**:
- Base framework endpoints handle RBAC, `logined_userorgid`, `confidential_fields` redaction, and DBFilter parsing correctly
- Custom scripts often violate `.dspy` file conventions (imports, dict access patterns, etc.)
- Replacing stable framework code with custom scripts creates maintenance burden — "today this way, tomorrow that way" makes the system unmaintainable

**When to propose custom scripts**: Only when there is a genuine special requirement that the CRUD framework cannot handle. Even then, **discuss the approach first** before implementing — get confirmation that the deviation is necessary and the proposed solution is acceptable. The user has zero tolerance for unsolicited replacements of stable framework code.

### CRITICAL: Hand-written list .dspy SHADOWS framework auto-generated endpoints

When a module has CRUD JSON definitions in `json/` (e.g., `table_list.json`), the Sage CRUD framework (via `xls2ui` during `build.sh`) **auto-generates** list endpoints inside subdirectories (e.g., `wwwroot/table_list/get_table_list.dspy`). If you also create a hand-written `wwwroot/api/get_{table}_list.dspy`, it **shadows** (overrides) the framework-generated one. This causes:
- **500 errors**: hand-written code may apply filters (e.g., `org_id`) on fields that don't exist in the table
- **403 errors**: hand-written files bypass the framework's built-in RBAC and `logined_userorgid` handling
- **Silent data leakage**: hand-written code misses `confidential_fields` redaction

**Rule**: If `json/{alias}_list.json` exists, do NOT create `wwwroot/api/get_{table}_list.dspy`. Only create hand-written `.dspy` for:
- Custom business logic endpoints (create, update, delete actions beyond simple CRUD)
- Client-specific dropdown data APIs that return `[{value, text}]` format (these are NOT list endpoints — they serve Form dropdowns)

**How to detect**: If a CRUD list endpoint returns 500 or 403, check whether both a `json/` definition AND a hand-written `api/get_*_list.dspy` exist for the same table. If both exist → DELETE the hand-written one.

### CRITICAL: CRUD-Generated wwwroot Directories Must NOT Be Git-Tracked

The `xls2ui` tool generates subdirectories in `wwwroot/` for each CRUD definition (e.g., `wwwroot/product_list/`, `wwwroot/llm/`). Each contains `index.ui`, `get_*.dspy`, `add_*.dspy`, etc. These are **auto-generated build artifacts** and must NOT be committed to git.

**Required .gitignore entries for any module using CRUD:**
```
# CRUD definition directories (auto-generated by Sage platform)
wwwroot/llm/
wwwroot/llmusage/
wwwroot/llm_api_map/
# ... etc for each CRUD alias
```

**WRONG**: `git add wwwroot/` — this pulls in auto-generated CRUD directories
**CORRECT**: `git add wwwroot/index.ui wwwroot/api/ wwwroot/*.js wwwroot/*.css` — add only hand-written files

**Symptom**: User will reject or correct you if you commit CRUD-generated directories. These files are regenerated by `build.sh` on every deployment.

### CRITICAL: Selective `git add` in Module Repos

Never use `git add -A` or `git add wwwroot/` in Sage module repos. Module repos have multiple categories of generated content that must NOT be committed:
- `build/` and `*.egg-info/` — Python build artifacts
- `__pycache__/` — Python bytecode cache
- CRUD-generated wwwroot subdirectories — auto-generated by `xls2ui`
- `*.swp`, `*.swo` — vim/editor swap files (add to .gitignore)

**Always use selective `git add`:**
```bash
# For Python package changes:
git add mymodule/__init__.py mymodule/init.py

# For frontend changes (only hand-written files):
git add wwwroot/index.ui wwwroot/api/ wwwroot/*.js wwwroot/*.css

# For CRUD JSON changes:
git add json/ models/
```

### CRITICAL: Manual Symlink for New Files During Development
When you create new `.ui`, `.dspy`, `.js`, or `.css` files in a module's `wwwroot/` during development, they are NOT automatically available until linked:

```bash
# Link individual files:
cd /path/to/sage/wwwroot/module_name/
ln -sf /path/to/module/wwwroot/new_page.ui .
ln -sf /path/to/module/wwwroot/api/new_api.dspy api/
ln -sf /path/to/module/wwwroot/new_script.js .

# Or re-run build.sh to refresh all links
```

**IMPORTANT**: `.js` and `.css` files must be linked at the module wwwroot **root** level (not in subdirectories like `scripts/` or `styles/`). Sage's `header.tmpl` only scans for JS/CSS at the root of each linked wwwroot directory.

**Symptom**: 500 error with `fpath is None` and `invalid path` in server logs — the file exists in the module but isn't linked to Sage's wwwroot.

### CRITICAL: build.sh SAGE_ROOT detection
When writing `build.sh` for a standalone module repo (e.g. `~/repos/mymodule/`), the script is NOT nested under Sage's directory. Do NOT assume `$SCRIPT_DIR/../..` points to Sage root. Use a search pattern:

```bash
for candidate in "$SCRIPT_DIR/../.." "$HOME/repos/sage" "$HOME/sage"; do
    if [ -d "$candidate/wwwroot" ] && [ -d "$candidate/py3/bin" ]; then
        SAGE_ROOT="$(cd "$candidate" && pwd)"
        break
    fi
done
```

## Sage System Integration

New modules must be wired into the Sage system in FOUR places before they appear in the UI.

### 1. `app/sage.py` — Module Loading

Add the import at the top of `~/repos/sage/app/sage.py` (near other module imports):
```python
from mymodule.init import load_mymodule
```

Add the load call in the `init()` function (near other load calls):
```python
load_mymodule()
```

### 2. `build.sh` — Installation Loop

Add the module name to the installation loop in `~/repos/sage/build.sh`:
```bash
for m in appbase rbac ... existing_modules mymodule
do
    echo "install $m module..."
    cd $cdir/pkgs
    git clone https://git.opencomputing.cn/yumoqing/$m
    cd $m
    $cdir/py3/bin/pip install .
    ...
done
```

### 3. `load_path.py` — RBAC Permissions

Add entries to the `paths=""` string in `~/repos/sage/load_path.py`:
```
/mymodule logined
/mymodule/index.ui logined
/mymodule/menu.ui any
/mymodule/api/endpoint.dspy logined
/mymodule/scripts/file.js any
```

**Permission rules:**
- `any` — no login required. Use for: `menu.ui`, static JS/CSS files, public resources.
- `logined` — requires authenticated user. Use for: `.ui` pages, `.dspy` APIs.
- `owner.superuser` / `owner.operator` — role-restricted admin features.

After editing `load_path.py`, run: `cd ~/repos/sage && ./py3/bin/python load_path.py`

**Note:** `load_path.py` is the centralized declarative source of truth. `set_role_perm.py` is for ad-hoc fixes only and does NOT survive a fresh database restore.

See `references/independent-web-app-pattern.md` for building standalone ahserver web applications (own venv, own port, own config.json) — distinct from Sage sub-modules.
See `references/rbac-init-pattern.md` for bootstrapping organization types, roles, users, and role-permission assignments for independent apps.
See `references/read-only-module-pattern.md` for the complete read-only module template.
See `references/test-env-module-deployment.md` for deploying modules from ~/test/ into Sage test environment (pip install, symlink, RBAC, restart, git push).
See `references/public-endpoint-cron-pattern.md` for exposing dspy endpoints without auth for cron/background jobs.
See `references/hub-dashboard-pattern.md` for the hub entry page pattern (stat cards + nav buttons + urlwidget content area) for business modules with CRUD + management views.
See `references/registry-table-pattern.md` for the registry/mapping table + dynamic table routing pattern.
See `references/llmage-openai-endpoint-pattern.md` for the pattern of adding OpenAI-compatible API endpoints to the llmage module (/v1/chat/completions, /v1/video/generations, /v1/image/generations).
See `references/llmage-data-model.md` for the llmage data model key tables, JOIN patterns, which fields belong to which table (apiname is in llm_api_map NOT llm), and subquery alias scoping pitfalls.
See `references/api-integration-pattern.md` for the uapi/llmage 4-table API integration pattern (no code changes needed).
See `references/tencent-hunyuan-api.md` for Tencent Hunyuan model specifications, pricing, and Sage integration notes.
See `references/external-api-integration-module-pattern.md` for modules that integrate with third-party APIs.
See `harnessed-module-development` skill → `references/vendor-callback-pattern.md` for vendor webhook/callback endpoint implementation.
See `references/rbac-tables.md` for the full RBAC table chain (organization→role→userrole→rolepermission→permission) and the correct init order for org/role/user creation in independent apps.
See `references/volcengine-ark-api.md` for Volcengine Ark real person portrait asset API — AK/SK signing, asset group/asset workflow, and hybrid module pattern (own client + uapi config).
See `references/downapp-proxy-api-pattern.md` for the Downapp user proxy API pattern — centralized vendor config table, org-to-resource mapping table, ownership validation, and multi-vendor factory design.
See `references/configurable-rule-engine-pattern.md` for the registry-based dynamic rule engine pattern — when modules need user-configurable validation rules (discounts, permissions, workflows).
See `references/platform-cross-org-table-pattern.md` for cross-org (platform-level) table design — tables that bridge multiple organizations without resellerid isolation, unified supplier mapping (internal + external), and menu grouping with separators.
See references/cache-control-pattern.md for the config.json `module_cache` toggle pattern — enable/disable in-memory caching per module via config.json.
See `references/sage-deployment-pitfalls.md` for site-packages sync, DSPY async limitations, CRUD before_insert hooks, and browser testing requirement.
See `references/pipeline-app-deployment-pitfalls.md` for pipeline-app umbrella repo recovery, bricks dist build, dspy return-outside-async-with, and RBAC static file permissions.
See `references/feature-enablement-three-layer-pattern.md` for systematically unlocking features gated behind hardcoded UI/dspy/backend limits.
See `references/cross-database-query-pattern.md` for the cross-database JOIN pattern — when a .dspy queries tables from another module's database using fully-qualified names (e.g., `product_management.product` from the `discount` database).
See references/social-platform-pattern.md for content/social module patterns — counter updates (R+U), toggle likes, nested comments (parent_id+reply_count), view counting, paid downloads, media type classification, and paginated feed APIs.
See `references/ai-compute-service-pattern.md` for deploying GPU-accelerated AI services (4090 server → ahserver HTTP API with nginx IP whitelist → Sage llmage+uapi integration → Hermes skill consumption).
- **IntegrityError handling**: See `references/dspy-error-handling.md` for error handling, UPSERT, and form `id` management patterns in .dspy.
See `references/dspy-nonetype-defense.md` for mandatory try/except wrapper patterns on all DSPY-called functions.

See `references/dspy-pitfalls.md` for return format, sqlor API signatures, param cleaning patterns, return-inside-async-with pitfalls, dbname naming, and $placeholder escaping.

See `references/module-extraction-pattern.md` for extracting embedded modules from umbrella repos to independent repos with `git subtree split`.

See `references/bricks-dist-build.md` for the mandatory bricks `dist/` build + symlink step after fresh clone/restore.
See `references/crud-list-query-optimization-pattern.md` for optimizing slow CRUD list queries — exclude TEXT/LONGTEXT fields from SELECT, add composite indexes on (filter_field, sort_field).
See `scripts/dspy_audit.sh` for the pre-commit dspy audit script — run before any commit touching .dspy files.
See `scripts/module_compliance_audit.py` for the batch module compliance audit — checks init/data.json format, load_path.py existence, and model codes parentid= compliance across multiple module repos.
See `references/sage-config-and-db-export.md` for parsing Sage's non-standard config.json (bare objects), RC4 password decryption, table discovery from models/*.json, and mysqldump export patterns.
See `references/i18n-merge-pattern.md` for the i18n merge workflow — consolidating module translations into `wwwroot/i18n/` via `scripts/merge_i18n.py` (updated: uses build.sh module list, ~/repos paths).

See `references/popup-form-refresh-pattern.md` for the pattern of refreshing parent page content after a Form inside a PopupWindow submits successfully (target: app.sage_main_content).
See `references/bricks-tabular-code-field-pattern.md` for bricks Tabular code field requirements — static `data` arrays vs dynamic `dataurl`+`valueField`/`textField` patterns, and common `undefined.length` errors.
See `references/feature-page-addition-checklist.md` for the 7-step workflow when adding a new feature page to an existing Sage module (dspy → ui → menu → RBAC → CSS → commit per repo → deploy).
See `references/supplychain-attachment-pattern.md` for filemgr attachment integration with drag-drop Form upload pattern (Form submit_url + UiFile multiple).
See `references/marketing-discount-setting.md` for marketing plan product discount override with base-comparison validation and flag pattern.

See `references/toolbar-ensure-pattern.md` for the one-click toolbar check-or-create pattern — `actiontype: "dspy"` + `params_mapping` to ensure a related record exists.

See `references/subtable-fk-auto-populate.md` for the pattern of auto-populating foreign keys in subtable add forms (editexclouded + new_data_url query params).

## Multi-Tenant Reseller Module Pattern

When different resellers (organizations) need their own independent data structures (category trees, product definitions, configs), use this org_id-scoped isolation pattern:

### Core Principle
Every table, CRUD definition, and API endpoint must be scoped by `org_id`. Different resellers have completely independent data — they never share categories, products, or configs.

### Table Design
- Every table MUST have `org_id VARCHAR(32) DEFAULT '0'` field
- Composite unique indexes use `(org_id, business_key)` — e.g., `UNIQUE(org_id, product_code)`
- Foreign key lookups must include `org_id` in JOIN conditions: `JOIN t2 ON t1.cat_id = t2.id AND t1.org_id = t2.org_id`

### Dynamic Attributes: `product_type` + `extra_json` (NOT physical table routing)
- Use a `product_type` field as a **string identifier** (not a physical table name)
- Store reseller-specific product attributes in `extra_json` (TEXT/LONGTEXT column)
- Each reseller defines their own structure inside `extra_json` — no schema changes needed
- Category tables store `product_type` and `product_type_title` for display, not `product_table_name`
- Standardized APIs (`product_detail`, `product_use`) parse `extra_json` and return `extra_parsed`

### CRUD Definition Requirements
- `logined_userorgid: "org_id"` on ALL list CRUD definitions — enables automatic org-scoped filtering
- Category dropdown dataurl must point to an API that filters by current user's org_id
- `subtables[].url` references must use `../` prefix and respect org boundaries

### .dspy API Requirements
- **Get org_id**: use `await get_userorgid()` (registered global async function by ahserver processorResource.py) — do NOT use `ServerEnv()` in .dspy files
  - Correct: `org_id = (await get_userorgid()) or '0'`
  - Correct (with optional param override): `org_id = params_kw.get('org_id', None) or (await get_userorgid()) or '0'`
  - WRONG: `env = ServerEnv(); org_id = getattr(env, 'orgid', None) ...`
- ALL SQL queries MUST include `org_id` in WHERE clauses
- Create/update/delete operations MUST verify the record belongs to current org before modifying
- Parent-child relationships (category trees) MUST verify parent belongs to same org
- Category dropdown APIs (`category_options.dspy`) must filter: `WHERE org_id = ${org_id}$`

### Operator Configuration Pattern
- `product_type_config` table: `(org_id, operator_id, category_id, config_name)` unique key
- Allows each operator within a reseller to define their own config for each category
- `config_json` TEXT field stores flexible configuration; parse to `config_parsed` on retrieval
- Query with fallback: `WHERE operator_id = ${user_id}$ OR operator_id = '0'` (0 = global default)

### Pitfalls
- **Do NOT use `product_table_name` pointing to physical tables** — this breaks when different resellers have different structures. Use `product_type` (identifier) + `extra_json` instead.
- **Do NOT preset global category data in `init/data.json`** — categories are managed per reseller. Init data should only contain appcodes/encoding definitions.
- **JOIN without org_id = data leak** — `LEFT JOIN product_category pc ON p.category_id = pc.id` without `AND p.org_id = pc.org_id` will return categories from other resellers if IDs happen to overlap.
- **Unique index without org_id = cross-reseller conflicts** — `UNIQUE(product_code)` prevents two resellers from using the same code. Must be `UNIQUE(org_id, product_code)`.
- **DBPools() is a singleton (`@SingletonDecorator`)** in Python module code — calling `DBPools()` returns the already-configured instance from startup. Do NOT write `db.databases = config.databases` or `DBPools(config.databases)`. Simply use `DBPools().sqlorContext(dbname)`.
- **In Python module files (*.py), get org_id via `ServerEnv()`**: `env = ServerEnv(); org_id = getattr(env, 'orgid', None) or getattr(env, 'org_id', '0')`. This is different from .dspy files which use `await get_userorgid()`.

## Multi-Module Changes: PR Isolation

When making changes across multiple Sage modules (UI redesign, stat cards, bug fixes, etc.), **each module must use a separate feature branch and independent PR**. Do NOT combine multi-module changes into a single branch.

**Correct pattern:**
```
accounting:     feat/dataviz-accounting  → PR #1
llmage:         feat/dataviz-llmage      → PR #2
dashboard:      feat/dataviz-users       → PR #3
```

**Wrong:** One branch `feat/all-changes` touching accounting/, llmage/, dashboard/ repos.

This keeps review focused, allows partial merges, and prevents blocking unrelated modules on a single review cycle.

## Cross-Module Changes: Where to Make Changes

**All modules are independent git repos at `~/repos/<module>/`.** Make changes and commit directly there. There is no `sage/pkgs/` directory — it was deleted in May 2026.

```
# CORRECT: modify in the module's own repo
cd ~/repos/uapi/
# ... make changes ...
git add ... && git commit -m "..." && git push
```

For cross-module changes (e.g., llmage modifying JOINs that involve uapi tables), check each affected module repo separately. After every round of changes, verify no stale patterns remain:

```bash
grep -rn 'old_pattern' ~/repos/<module>/ --include='*.py' --include='*.dspy'
```

**Restart Sage** after all code changes to reload modules from disk.

## Database Schema Migration

See `references/database-migration-pattern.md` for the complete workflow when modifying database schemas in production modules:
- Adding new tables / changing relationships (1:N → M:N)
- Writing idempotent migration scripts with safety checks
- Setting RBAC permissions for new CRUD endpoints
- Safe column removal procedures

### Intermediate Table Removal

When removing an intermediate/junction table (e.g., `uapiset` between `upapp` and `uapi`), the process requires:

1. **Add direct join column**: `ALTER TABLE child ADD COLUMN parent_id ...`
2. **Move fields to parent**: Move any fields that were on the intermediate table (e.g., `auth_apiname`) to the parent table (`upapp`)
3. **Update model JSON**: Remove old FK field, add new FK field, update indexes, remove codes references to the removed table
4. **Update ALL Python code**: Remove JOINs to the intermediate table in every `.py` file. Check the module repo at `~/repos/{mod}/` and any other modules that reference the table.
5. **Delete obsolete files**: Remove `models/{intermediate}.json` and `json/{intermediate}.json`
6. **Data migration**: Handle shared records (multiple parents sharing same children through the intermediate table). See `~/repos/uapi/scripts/migrate_uapi_upappid.py` for the pattern.
7. **Verify**: `grep -rn 'intermediate_table' ~/repos/{mod}/ --include='*.py'` — should return nothing.

See `references/intermediate-table-removal-pattern.md` for the complete detailed pattern with SQL examples.
See `references/uapi-uapiset-removal-detail.md` for a concrete session example (uapiset removal from uapi module).

## RBAC Permission Setup

**Preferred approach: Per-module `scripts/load_path.py`**

Each business module should own its own `scripts/load_path.py` that registers all its RBAC paths. This keeps permissions self-contained and auditable per module.

**Maintenance rule**: 每次代码变更如有新 path 出现，需同步更新此脚本

The script should:
1. Find the Sage root directory automatically
2. Define paths by role tier (`any`, `logined`, role-specific)
3. Call `set_role_perm.py` for each path

See `references/per-module-rbac-pattern.md` for the complete script template.
See `references/per-module-load-path-pattern.md` for a concrete script template with role-tier separation.

**NOTE**: Some modules (like llmage) use the **central** `load_path.py` in `~/repos/sage/load_path.py` instead of a per-module script. Before creating a new `scripts/load_path.py`, check if the module already has one or if it uses the central approach.

### CRITICAL: load_path.py wildcards are FORBIDDEN

**NEVER use `%` or `*` wildcards in any load_path.py file.** This is a hard policy rule, not a reliability suggestion. All RBAC paths must be listed explicitly — every `.dspy` file, every CRUD subdir file, every `.ui` file, every image.

**Correct approach:**
```python
PATHS_LOGINED = [
    f"/{MOD}/api/endpoint1.dspy",
    f"/{MOD}/api/endpoint2.dspy",
    f"/{MOD}/api/get_pricing_display.dspy",
    # ... list every .dspy file explicitly
    f"/{MOD}/table_alias",
    f"/{MOD}/table_alias/index.ui",
    f"/{MOD}/table_alias/get_table_alias.dspy",
    f"/{MOD}/table_alias/add_table_alias.dspy",
    f"/{MOD}/table_alias/update_table_alias.dspy",
    f"/{MOD}/table_alias/delete_table_alias.dspy",
]
```

**CRUD subdirectory standard pattern** (5 files per alias, generated by xls2ui from json/*.json):
```python
f"/{MOD}/{alias}",
f"/{MOD}/{alias}/index.ui",
f"/{MOD}/{alias}/get_{alias}.dspy",
f"/{MOD}/{alias}/add_{alias}.dspy",
f"/{MOD}/{alias}/update_{alias}.dspy",
f"/{MOD}/{alias}/delete_{alias}.dspy",
```

**Maintenance rule**: When adding new `.dspy` files, you MUST also add them to `PATHS_LOGINED` (or `PATHS_ANY`) explicitly. When removing files, remove the corresponding entries.

**Cross-module audit**: When fixing wildcards in one module, check ALL modules (`find ~/repos -name "load_path.py"`) for remaining wildcards with `grep -rn '%' --include="load_path.py"`. All 9+ modules should be wildcard-free.

### 4. `global_menu.ui` — Menu Entry

Add a menu item to `sage/wwwroot/global_menu.ui` inside the `items` array:

```json
,{
    "name": "modulename",
    "label": "模块名称",
    "icon": "fa fa-icon-name",
    "url": "{{entire_url('/modulename/index.ui')}}",
    "target": "app.sage_main_content"
}
```

**CRITICAL**: Do NOT wrap module menu items in `{% if get_user() %}` or role conditionals. Sage's dynamic menu loading does not work — items inside conditionals are never rendered. Place all module items unconditionally. RBAC (`load_path.py`) handles access control, not menu visibility.

Usage:
```bash
cd ~/repos/sage
./py3/bin/python ~/repos/<module>/scripts/load_path.py
```

The script must register **both** directory paths and file paths for CRUD aliases:
```python
# Directory path (auto-matches index.ui)
set_role_perm.py "logined" "/module/crud_alias"
# File path (direct access)
set_role_perm.py "logined" "/module/crud_alias/index.ui"
```

See `references/per-module-rbac-pattern.md` for the complete script template.
See `references/per-module-load-path-pattern.md` for a concrete script template with role-tier separation.

**Legacy approach: Central `load_path.py`**

Add entries to the `paths=""` string in `~/repos/sage/load_path.py`:
```
/module_name logined
/module_name/index.ui logined
/module_name/menu.ui any
/module_name/api/xxx.dspy logined
/module_name/xxx.js any
```

- **logined** — pages and API endpoints that require authentication
- **any** — static resources (menu.ui, .js, .css, images) that must load without auth

### Steps
1. Edit `~/repos/sage/load_path.py` — add entries to the `paths` string in the appropriate location (alphabetically by module, or grouped by feature)
2. Run on the server: `cd ~/repos/sage && ./py3/bin/python load_path.py`
3. Restart Sage: `./stop.sh && ./start.sh`

**Note**: `set_role_perm.py` is for ad-hoc fixes only and does NOT survive a fresh database restore. Both `set_role_perm.py` and `load_path.py` must use explicit paths — no wildcards in either.

**IMPORTANT**: Do NOT manually reference `.css`/`.js` files in `.ui` widgets with `<link>` or `<script>` tags. ahserver auto-serves these files from wwwroot/. However, they still need RBAC registration in `load_path.py` (see `references/per-module-rbac-pattern.md`).

## Integrating a New Module into Sage

Beyond `pip install` and wwwroot symlinks, a new business module requires FOUR changes in the Sage repo:

1. **app/sage.py** — Add import and load call:
   ```python
   from modulename.init import load_modulename
   # ... in init():
   load_modulename()
   ```

2. **build.sh** — Add module name to the installation loop:
   ```bash
   for m in ... existing_modules modulename
   ```

3. **load_path.py** — Add RBAC permission entries (see RBAC Permission Setup section above)

4. **global_menu.ui** — Add menu entry in `sage/wwwroot/global_menu.ui`:
   ```json
   ,{
       "name": "modulename",
       "label": "模块名称",
       "icon": "fa fa-icon-name",
       "url": "{{entire_url('/modulename/index.ui')}}",
       "target": "app.sage_main_content"
   }
   ```

All four are required. Missing any one causes: (1) module functions not registered, (2) module not installed on fresh deployments, (3) 403 Forbidden on page access, (4) menu item not visible.

### 5. Execute per-module `scripts/load_path.py` (if it exists)

If the module has its own `scripts/load_path.py`, run it to register permissions via `set_role_perm.py`:
```bash
cd ~/repos/sage
./py3/bin/python ~/repos/<module>/scripts/load_path.py
```

**CRITICAL: Dual-layer RBAC pitfall.** Some modules (cpcc, product_management, pricing, etc.) have per-module `scripts/load_path.py` files that call `set_role_perm.py` directly. However, **`set_role_perm.py` has been failing** due to `ModuleNotFoundError: No module named 'appPublic.event_dispatcher'` in the virtual environment. This means per-module permissions may never reach the database.

Therefore, **always ALSO add entries to the central `~/repos/sage/load_path.py`**. The central `load_path.py` uses a different code path and is the reliable fallback. If only the per-module script exists and it fails silently, the module will show in the menu but every click returns 403.

**Verification:** After running load_path.py, confirm permissions exist in the database:
```bash
grep -c '/modulename' ~/repos/sage/load_path.py  # should be > 0
```

### CRITICAL: global_menu.ui Menu Items — NO Conditional Blocks

**Do NOT place module menu items inside `{% if get_user() %}` or role-check conditionals.** Sage's dynamic menu loading logic does not work — items inside conditionals are never rendered. All business module menu items must be placed at the top level of the `items` array, unconditionally.

The only exception: the `dashboard` item (always visible by design) and role-restricted items that the user explicitly wants hidden from certain roles. When in doubt, make it unconditional.

**WRONG**:
```jinja2
{% if get_user() %}
,{ "name": "mymodule", "label": "我的模块", ... }
{% endif %}
```

**CORRECT**:
```json
,{ "name": "mymodule", "label": "我的模块", ... }
```

Access control is enforced by RBAC (`load_path.py`), not by hiding menu items. If a user clicks a module they don't have permission for, they get a 403 — which is the correct behavior.

## Development Workflow

### CRITICAL: Code Modification Workflow — Repos First, Test Second

**All code modifications MUST follow this sequence:**
1. **Modify source code in `~/repos/` directory first** — this is the canonical source
2. **Git commit immediately** in the repos directory
3. **Sync to test environment** (`~/test/` or deployment directory) for verification
4. **Never modify test environment directly** — changes in `~/test/` are lost and untracked

**Why this matters:**
- `~/repos/` is version-controlled — changes survive and can be reviewed
- `~/test/` is ephemeral deployment — direct edits there vanish on redeploy
- User has zero tolerance for "I fixed it in test but didn't update repos"

**CRITICAL: scp/sed directly to servers is FORBIDDEN.** Never use `scp` to copy files or `sed` to edit files directly on deployment servers (tokentest, production). Always go through the git workflow: local repos → commit → push → server git pull → pip install. Direct edits bypass version control, create unreviewable changes, and will be overwritten on the next deploy. The one exception is one-off diagnostic commands (grep, mysql queries, log inspection) that don't modify files.

**Wrong:**
```bash
# Modifying test directly
cd ~/test/pipeline-app
# ... make changes ...
# (changes lost on next deploy, no git history)
```

**Also wrong:**
```bash
# scp/sed directly to server
scp local_file apitest@server:/remote/path/
ssh apitest@server "sed -i 's/old/new/' /remote/file"
```

**Correct:**
```bash
# Modify repos first
cd ~/repos/pipeline
# ... make changes ...
git add scripts/load_path.py
git commit -m "fix: remove hardcoded Sage paths"
git push

# Then sync to test
cp ~/repos/pipeline/scripts/load_path.py ~/test/pipeline-app/pipeline/scripts/
```

See `references/umbrella-repo-module-extraction.md` for extracting embedded modules from umbrella (monorepo) apps into independent repos with preserved git history.

See `references/bulk-module-creation-pattern.md` for the complete ordered creation sequence, bulk .dspy API generation script, init.py function templates, auto-generated code patterns, cascade delete patterns, and CRITICAL subagent validation checklist.
See `references/app-level-wwwroot-pattern.md` for the critical rule that applications must have a unified app-level `wwwroot/` directory — all frontend files go there, NOT scattered across module directories. Modules keep their Python code but move static files to the app's wwwroot.

1. Create module directory structure with all required subdirectories
2. Create table definitions in `models/` directory
3. Create CRUD definitions in `json/` directory
4. Implement backend functions in `{module}/init.py` — one async function per CRUD op per table, registered via `load_{module}()` with ServerEnv
5. Generate `wwwroot/api/*.dspy` files — thin wrappers delegating to init.py functions (use bulk generation script from reference)
6. Develop frontend using bricks-framework in wwwroot/ (index.ui + menu.ui first, CRUD UI auto-generated by xls2ui)
7. Add initialization data to init/data.json if needed
8. Configure pyproject.toml and write build.sh
9. Write README.md with Sage integration steps
10. Wire module into Sage system: `app/sage.py` (import + load), `build.sh` (install loop), `load_path.py` (RBAC permissions), `global_menu.ui` (menu entry — unconditional, NOT inside `{% if %}`), execute per-module `scripts/load_path.py` if it exists — see **Sage System Integration** section
11. Create skill/ directory with SKILL.md, references/, assets/, and scripts/ subdirectories

## Parameter Access: .py vs .dspy/.ui

**Rule**: Parameter access differs between Python functions and dspy/ui templates.

### In .dspy and .ui templates
Use `params_kw` directly — it is injected as a global:
```python
# .dspy file
discountid = params_kw.get('discountid')
```
```jinja2
<!-- .ui file -->
{{params_kw.discountid}}
```

### In Python (.py) functions called from Jinja2
Use `request._run_ns.params_kw` — the `request` object passed to ServerEnv-registered functions holds params in its `_run_ns` namespace:
```python
async def my_func(request):
    env = request._run_ns
    val = (request._run_ns.params_kw or {}).get('key', '')
```

**Pitfall — wrong attribute names.** Do NOT use:
- `request._run_env` → AttributeError (does not exist, correct name is `_run_ns`)
- `request._run_ns.param_kw` (singular `param`) → wrong, the attribute is `params_kw` (plural `params`)
- `request._params_kw` → may be empty at Jinja2 render time
- `request.rel_url.query` or `request.query` → unnecessary, params already parsed into `_run_ns.params_kw`

### sqlExe: No Hardcoded ORDER BY / LIMIT / OFFSET in SQL

**Rule**: Do NOT hardcode `ORDER BY`, `LIMIT`, or `OFFSET` in SQL strings. sqlor handles these via `sqlPaging()`:
```python
# WRONG — hardcoded ORDER BY, LIMIT, OFFSET
sql = "SELECT * FROM t ORDER BY name LIMIT ${pagerows}$ OFFSET ${offset}$"

# CORRECT — use sor.sqlPaging() which handles pagination automatically
ns = {"page": 1, "sort": "name", "order": "desc", "rows": 20}
recs = await sor.sqlPaging("SELECT * FROM t", ns)
# Returns {"total": N, "rows": [...]}
```

**Pitfall: sqlExe returns DictObject — NOT dict-convertible**
sqlor's `sqlExe()` returns `DictObject` instances. `dict(r)` fails with `ValueError: dictionary update sequence element #0 has length 1; 2 is required`. Access attributes directly:
```python
# WRONG — DictObject not iterable as (k,v) pairs
rows = [dict(r) for r in recs]

# CORRECT — access attributes individually
rows = [{"id": r.id, "name": r.name} for r in recs]
```

### dspy Database Access: get_sor_context NOT DBPools().sqlorContext

**Rule**: In `.dspy` files, use `get_sor_context(env, 'module_name')` which routes to the correct database per the application's `module_dbname` config. Do NOT use `DBPools().sqlorContext("hardcoded_db")`:
```python
# WRONG — hardcoded db name, tied to one application
db = DBPools()
async with db.sqlorContext("pipeline") as sor:
    ...

# CORRECT — routes to correct db per application config
env = request._run_ns
async with get_sor_context(env, "tenant") as sor:
    ...
```
This way, `get_sor_context(env, "tenant")` resolves to `pipeline` database in pipeline-app and `sage` database in Sage.

## Bricks HBox Column Widths: cwidth vs width%

**Critical — `cwidth` in HBox is NOT proportional.** In bricks HBox (plain flex containers), `cwidth` sets a fixed pixel width: `charsize × cwidth` (e.g., cwidth=3 ≈ 42px). This is different from Tabular's DataRow where cwidth=10 is the default.

For proportional column widths in HBox, use `width` with percentage values:
```json
// WRONG — fixed 42px/70px columns, won't scale
{"widgettype": "HBox", "subwidgets": [
    {"widgettype": "Text", "options": {"cwidth": 3}},
    {"widgettype": "Text", "options": {"cwidth": 5}}
]}

// CORRECT — proportional 22%/34% columns
{"widgettype": "HBox", "options": {"width": "100%"}, "subwidgets": [
    {"widgettype": "Text", "options": {"width": "22%"}},
    {"widgettype": "Text", "options": {"width": "34%"}}
]}
```

This applies to any bricks HBox, including hand-built list rows. Only Tabular's DataRow handles cwidth proportionally.

## Git Initialization & First Push for New Modules

When creating a brand-new module repo (not yet on Gitea), follow this exact sequence:

### Pitfall: Gitea "Push to create" is DISABLED
New repos MUST be created on the Gitea web UI (git.opencomputing.cn) before `git push`. SSH push-to-create returns: `Gitea: Push to create is not enabled for users. fatal: Could not read from remote repository.`

**Fix**: Create the empty repo on Gitea web UI first, THEN push locally.

### Pitfall: Auto-generated README causes rebase conflict
Gitea creates an initial README.md when you create the repo. Your local commit also has a README.md. First `git pull --rebase` triggers `CONFLICT (add/add): Merge conflict in README.md`.

**Fix**:
```bash
git checkout --ours README.md    # keep local version
git add README.md
GIT_EDITOR="true" git rebase --continue   # "true" avoids nano/editor hang
git push -u origin main
```

### Pitfall: `git rebase --continue` opens nano editor
In non-interactive terminal sessions, `git rebase --continue` opens nano which hangs waiting for stdin. Always prefix with `GIT_EDITOR="true"`.

### Standard first-push sequence
```bash
cd ~/repos/{module}
# 1. Write .gitignore FIRST (see references/module-gitignore.md for template)
# 2. Init and commit
git init && git add -A && GIT_EDITOR="true" git commit -m "feat: initial commit message"
# 3. Add remote
git remote add origin git@git.opencomputing.cn:yumoqing/{module}.git
# 4. Pull, resolve conflicts, push
git pull origin main --rebase 2>&1
# If conflict: git checkout --ours README.md && git add README.md && GIT_EDITOR="true" git rebase --continue
git push -u origin main
```

### CRITICAL: Never Fabricate Pricing or Config Data

**Pricing data (元/次, 元/秒, 元/token, etc.) must come from official vendor sources only.** Guessing, estimating, or pulling from outdated references is a production incident. The user explicitly framed this as a "生产事故".

**Rule**: If the official pricing page/documentation is not accessible (SPA, requires login, etc.), tell the user honestly that you cannot find it. Do NOT:
- Estimate from similar tier models
- Extrapolate from older versions
- Use memorized pricing that may be stale
- Pull from skill references written by earlier sessions (those were also guesses)

**How to get pricing**: Ask the user for screenshots or text from the official console/pricing page, or direct them to provide the vendor URL with accessible data.

**Applies to**: SQL pricing values, YAML price_factors, unit_prices in pricing_program_timing, unit_price in P4 migration scripts, and any other config where a wrong number has real billing consequences.

## Verification Steps

### CRITICAL: Self-Testing Before Commit — MANDATORY

**User has zero tolerance for untested code.** Before committing any module code, you MUST:

1. **Deploy to a test environment** — create a venv, install deps, start the app on a test port
2. **Exercise every endpoint** — curl or browser-test each `.dspy` API, each `.ui` page
3. **Check server logs** — `tail -f logs/*.log` during testing, catch import errors / AttributeErrors / 500s
4. **Cross-validate against working Sage code** — find real working `.dspy` files in `~/repos/sage/wwwroot/` and compare patterns

**Wrong approach**: Write code → commit → tell user to deploy and test
**Correct approach**: Write code → self-deploy → self-test → fix issues → commit only when verified

If you cannot set up a test environment, say so explicitly and do NOT commit.

### CRITICAL: dspy Batch Audit Before Commit — MANDATORY PRE-COMMIT GATE

**Run this audit BEFORE every commit that touches .dspy files.** This is not optional — it's a hard gate. If violations are found, fix them FIRST, then commit. Do NOT commit and fix later.

**Full list of pre-loaded symbols in dspy execution context** (available without any import):
- `json` (json.dumps, json.loads)
- `datetime` (datetime, date, timedelta)
- `time` (time.time, etc.)
- `os` (os.path, etc.)
- `debug`, `exception`, `error`, `info`, `warning`, `critical` (from appPublic.log)
- `DBPools` (from sqlor.dbpools)
- `get_sor_context` (from sqlor.dbpools)
- `DictObject` (from appPublic.dictObject)
- `get_user`, `params_kw`, `format_exc`, `password_encode`, `password_decode`
- `curDateString`, `timestampstr`, `getID`
- `functools.partial`
- `ahserver.filestorage.FileStorage`
- `RegisterFunction` (from appPublic.registerfunction)
- `DBFilter`, `ArgsConvert` (from sqlor.filter)
- All ServerEnv-registered functions (from every module's `load_*()`)

**Module-internal imports MUST go through load_*()**: `from mymodule.utils import foo` in dspy is not allowed. Instead, export `foo` via `load_mymodule()` in `init.py`:
```python
# init.py
from .utils import foo
def load_mymodule():
    env = ServerEnv()
    env.foo = foo
```

Pre-commit audit commands:
```bash
# Detect ALL import statements (only DBFilter is acceptable)
grep -rn "^import\|^from" wwwroot/ --include='*.dspy' | grep -v "sqlor.filter"
# The output should be EMPTY. If any lines appear, those imports must be
# removed (if pre-loaded) or moved to load_*() export (if module-internal).
```

**Workflow enforcement:**
1. Write/edit .dspy files
2. **Run audit script** → fix all violations
3. Re-run audit → confirm zero violations
4. Only then: `git add` → `git commit`

**Why this matters**: User will reject commits with violations. Fixing after commit creates noisy history and loses trust. The audit takes 5 seconds — do it every time.

See `references/dspy-batch-fix-pattern.md` for the systematic fix approach.

**Symptom**: `return data type error, <class 'NoneType'>` for a dspy that clearly has a return statement. Fix: move `return` OUTSIDE the async with block.

When a 500/NoneType error occurs with a stack trace, the FIRST file to inspect is the one named in the error. Do NOT chase tangential issues before examining the file at the top of the traceback.

**Real case**: 3 consecutive `currency_stats.dspy return data type error, NoneType`. Fixed init.py try/except, etl.py, SQL columns — all irrelevant. Root cause: JSON-format DSPY file where Python-script was expected.

**Rule**: Read the file in the error trace FIRST.

### CRITICAL: Database Name Consistency Check

When a module uses its own database (not `sage`), verify in THREE places:

1. **Python module `init.py`**: `DBNAME = "actual_db_name"` (not module name)
2. **App entry `global_func.py`**: `get_module_dbname()` returns correct DB name
3. **App entry `cms.py`**: `get_module_dbname()` override returns correct DB name

**Pitfall**: `DBNAME = "entcms"` when actual DB is `"ocai_cms"` → all queries fail with table-not-found.
**Pitfall**: `global_func.py` returns `'sage'` → dspy files query wrong database.

## Verification Steps

**MANDATORY: Test before commit. User will reject untested code.**

Before committing ANY code changes, you MUST:
1. **Build a test environment** (`~/test/` or dedicated test venv) — do NOT just read code and assume it works
2. **Actually run the code** — import the module, call the functions, hit the API endpoints
3. **Check all affected files** — don't just verify the file you changed, check every file that depends on it
4. **Verify dspy patterns** — read actual Sage dspy files (e.g., `gen_code.dspy`, callback dspy files) to confirm your patterns match production

**Common pitfalls that MUST be caught before commit:**
- `async with db.sqlorContext(dbname) as sor:` — dspy files MUST use this pattern, not `sor = DBPools().sqlorContext(dbname)`
- No `import` statements in dspy files (except `from sqlor.filter import DBFilter`)
- `return` instead of `print(json.dumps(...))`
- `getID()` not `uuid()`
- `await get_user()` not `get_user()`
- Python module DBNAME constants must match actual database name in config.json
- `get_module_dbname()` return value must match the app's actual database

**If you claim "tested" but didn't actually execute the code, the user WILL find out and you WILL lose trust.**

**Post-deployment testing is equally mandatory.** After deploying, immediately verify with curl + auth token:
```bash
# Login to get session cookie
curl -s -c /tmp/cookies.txt -X POST 'http://localhost:9180/rbac/user/login.ui' \
  -H 'Content-Type: application/x-www-form-urlencoded' -d 'username=test&password=test123'
# Test all affected endpoints
for path in /module/page /module/api/endpoint.dspy /imgs/icon.svg; do
  code=$(curl -s -o /dev/null -w '%{http_code}' -b /tmp/cookies.txt "http://localhost:9180$path")
  echo "$code $path"  # must be 200, not 403/500
done
```
Do NOT tell the user "可以测试了" without first running these checks yourself. The user's time is not for debugging your untested code.

- [ ] Module loads correctly via load_{modulename}() function
- [ ] Module imported and loaded in `app/sage.py`
- [ ] Module added to `build.sh` install loop
- [ ] RBAC permissions added to `load_path.py` and loaded via `python load_path.py`
- [ ] Menu item added to `sage/wwwroot/global_menu.ui` (NOT inside `{% if %}` conditionals)
- [ ] wwwroot symlinked to `sage/wwwroot/{modulename}/`
- [ ] All exposed functions work in frontend scripts
- [ ] Database operations follow sqlor specifications
- [ ] Frontend renders correctly with bricks-framework
- [ ] CRUD operations function as defined
- [ ] JSON config files contain `editable` section with correct `new_data_url`/`update_data_url`/`delete_data_url`
- [ ] JSON config file `entire_url()` references use `../` prefix for same-module CRUD aliases
- [ ] Initialization data loads properly
- [ ] Package builds successfully with pyproject.toml
- [ ] Skill documentation created in skill/ directory with complete SKILL.md
- [ ] Reference examples, assets, and supporting scripts organized appropriately