pipeline_core/skills_library/all/crud-definition-spec/SKILL.md

name	version	description	trigger_conditions
crud-definition-spec	1.0.0	定义 CRUD（json/*.json，根键 tblname+params）时必读——CRUD 定义格式、list/tree 两种视图、同表多逻辑才加 alias（目录=tblname，单逻辑不加）。不加载会产出不符合规范的 CRUD json、alias 滥用。表结构定义用 database-table-definition-spec。	User needs to create or modify CRUD definition files in JSON format Task involves generating CRUD configurations for the json directory Working with sqlor-database-module CRUD operations Need to determine between list vs tree view based on table relationships

CRUD Definition Specification

Overview

Standardized JSON format for CRUD operations integrating bricks-framework (frontend) and sqlor-database-module (backend). The framework auto-selects list vs tree view from table structure. Files live in the module's json/ directory and are generated into wwwroot/<table>/ UIs + dspy endpoints via xls2ddl.xls2crud.

View Type Determination

• Tree View: table has a self-referencing foreign key (one field points to another record's id in the same table)
• List View: all other tables (no hierarchical relationships)

Common Root Properties (Both View Types)

{
    "tblname": "table_name",        // Required: actual table name
    "alias": "optional_alias",      // Optional: multiple CRUD interfaces for same table
    "title": "Display Title",       // Optional: defaults to table title
    "params": { ... }               // Required: view-specific parameters
}

List View CRUD Specification

Complete Structure

{
    "tblname": "table_name",
    "alias": "optional_alias",
    "title": "Display Title",
    "params": {
        "sortby": ["field1 desc", "field2"],
        "logined_userorgid": "org_id_field",
        "logined_userid": "user_id_field",
        "data_filter": {
            "AND": [
                {"field": "field1", "op": "LIKE", "var": "filter_var1"},
                {"field": "field2", "op": "=", "var": "filter_var2"}
            ]
        },
        "confidential_fields": ["field1", "field2"],
        "editor": {
            "binds": [{
                "wid": "source_field_id",     // Required: source widget ID (field name)
                "event": "changed",           // Required: event type (typically "changed")
                "actiontype": "script",       // Required: action type ("script" for JS)
                "target": "target_field_id",  // Required: target widget ID
                "script": "// JS code"        // Required: JS script content
            }]
        },
        "browserfields": {
            "exclouded": ["id"],              // Optional: fields excluded from display
            "alters": {
                "field_name": {
                    "uitype": "code",         // Required: "code" = dropdown/select
                    "data": [{"value": "v1", "text": "Display Text"}],  // Required for uitype="code" (static)
                    // OR dynamic: "dataurl": "api/endpoint", "datamethod": "GET", "dataparams": {"param": "value"}
                }
            }
        },
        "editexclouded": ["readonly_field"],  // Optional: fields excluded from edit form
        "subtables": [{                       // Optional: foreign key relationships
            "field": "foreign_key_field",     // Required: FK field name
            "title": "Subtable Title",        // Optional: defaults to subtable title
            "url": "{{entire_url(subtable_alias)}}",  // Required when alias defined
            "subtable": "related_table_name"  // Required: related table name
        }]
    }
}

Tree View CRUD Specification

Complete Structure

{
    "tblname": "hierarchical_table",
    "alias": "optional_alias",
    "uitype": "tree",                          // Required: must be "tree" for tree view
    "title": "Display Title",
    "params": {
        "idField": "id",                       // Required: node ID field
        "textField": "display_field",          // Required: node display text field
        "sortby": ["field1 desc", "field2"],   // Optional: sort fields for tree nodes
        "confidential_fields": ["field1", "field2"],  // Optional: sensitive fields
        "browserfields": {"alters": {}},       // Optional: field attribute modifications
        "logined_userorgid": "org_id_field",   // Optional: org filtering field
        "logined_userid": "user_id_field",     // Optional: user filtering field
        "editable": true,                      // Required: true=editable, false=read-only
        "edit_exclouded_fields": ["system_field"],  // Optional: excluded from editing
        "parentField": "parent_id",            // Required: parent node reference field
        "subtables": [...]                     // Optional: same structure as list view
    }
}

Filter/Search Integration (data_filter)

data_filter in params follows the sqlor/filter.py DBFilter JSON format:

{"tblname": "llm", "params": {"data_filter": {"AND": [
    {"field": "model", "op": "LIKE", "var": "model_input"},
    {"field": "ppid", "op": "=", "var": "ppid_input"},
    {"field": "status", "op": "=", "const": "1"}
]}}}

Frontend behavior: With data_filter present, DataViewer adds a "搜索" toolbar button → PopupWindow with a Form. Fields are generated from the filter tree: each var becomes a form input; fields with browserfields.alters[field].uitype == "code" render as dropdowns. Customize via filter_labels: {"var_name": "显示名"} (labels), filter_label (button text), filter_title (popup title). No inline search form is rendered — filter UI is exclusively toolbar-triggered. Form submit sends data_filter (JSON string) + each var value as URL params → backend .dspy uses DBFilter.gen(ns) for the SQL WHERE clause.

Key rules:

• var: parameter name — value from user input in the popup form
• const: hardcoded value — no user input, does NOT generate a form field
• op supported: =, !=, >, >=, <, <=, IN, NOT IN, LIKE, NOT LIKE, IS NULL, IS NOT NULL
• Logical operators: AND (array, length ≥ 2), OR (array, length ≥ 2), NOT (single dict); OR/AND nestable
• const conditions are included in the sent data_filter JSON but generate no form inputs
• Empty filter values are excluded from request params (not sent as empty strings)

File Management Requirements

• All CRUD definition files must be stored in the json/ directory of the module
• Each table gets one or more JSON files (multiple if using aliases)
• Naming: {table_name}.json or {alias}.json — e.g. users → json/users.json; alias user_admin → json/user_admin.json

Key Implementation Notes

• Field exclusion: browserfields.exclouded hides fields in read-only/list view; editexclouded hides in edit forms (list view); edit_exclouded_fields hides in edit forms (tree view)
• Dynamic data loading: static data array (value/text pairs) | dynamic dataurl + datamethod + dataparams | cross-module data via references/cross-module-dataurl.md (multi-DB query pattern)
• Event binding: only in list view editor.binds, uses JavaScript for dynamic form behavior (e.g. cascading dropdowns); wid = source field, target = destination field
• Security: always specify confidential_fields for sensitive data; use logined_userorgid/logined_userid for data isolation; set editable: false for read-only tree views

Integration Requirements

• Frontend: bricks-framework; backend: sqlor-database-module; references table definitions in models/
• Follows module structure from module-development-spec skill
• Each data_url / editable URL must have a matching .dspy endpoint — see references/api-endpoint-patterns.md
• CRUD files are generated from JSON via xls2ddl.xls2crud — see references/xls2crud-generation.md

Validation Checklist

• View type correctly chosen (tree vs list based on table relationships)
• Required fields present for chosen view type
• tblname exactly matches a table defined in table definition
• params dict exists and is non-empty
• editable paragraph exists with new_data_url, update_data_url, delete_data_url
• data_url exists and points to a valid .dspy list endpoint
• Every field in browserfields.exclouded exists in the model's field list
• Every field in browserfields.alters keys exists in the model's field list
• Every field in editexclouded / edit_exclouded_fields exists in the model's field list
• All NOT NULL DEFAULT columns that aren't user-editable are in editexclouded (prevents "cannot be null" on submit)
• alters entries use uitype: "code" with dataurl (endpoint returns plain [{value,text}] array) or data array (static)
• alters entries with valueField/textField MUST also have uitype: "code" (else text mapping silently ignored — see Pitfall 33)
• subtables[].url uses {{entire_url('../alias')}} with ../ prefix, no wwwroot in path
• editor.binds[].actiontype is one of: urlwidget, method, script, registerfunction, event
• entire_url() arguments are quoted strings
• No forbidden root keys: tablename (use tblname), grid, form, name, type, components
• No Jinja2 control blocks in CRUD JSON ({% if %}, {% for %}). {{entire_url(...)}} in strings is OK. See references/crud-json-rules.md.
• File stored in correct json/ directory with {table_name}.json naming
• All referenced fields exist in table definition (models/ directory)
• Security fields properly configured (confidential_fields + browserfields.exclouded)
• Subtable refs valid: field and subtable keys present, subtable matches an existing table in models/, and a corresponding wwwroot directory or CRUD config exists
• Every data_url / editable URL has a matching .dspy endpoint file in wwwroot/api/
• If data_filter present, corresponding .dspy list endpoint uses DBFilter to parse it
• CRUD endpoint audit: Run scripts/verify-crud-endpoints.py from ~/repos/ to check all create/update/delete .dspy files for json.dumps wrapping, wrong return format, and sor.U argument count. Fix all failures before commit.

Forbidden Root Keys (DO NOT USE)

The following keys do not exist in the CRUD spec and will cause failures:

• tablename → use tblname
• grid → does not exist; CRUD files do NOT support fields/joins/select_fields (no SQL joins, no cross-table fields) — use browserfields.exclouded + alters
• form → does not exist — use editexclouded + alters with uitype
• name, type, components → not CRUD keys; such files are .ui files in wwwroot/, not CRUD JSON (see Pitfall 9)

Common Pitfalls

Pitfall 1: Root key is tblname, not tablename

"tablename": "users" is invalid — must be "tblname": "users".

Pitfall 2: CRUD files reference ONLY the base table

No SQL joins, select_fields, or cross-table field references. All fields in browserfields.exclouded, editexclouded, and alters must exist in the table definition (models/) for the table in tblname. Example: for financial_vouchers, reference only fields like contract_id, voucher_number, amount — never contract_number (that field is in the contract table).

Pitfall 3: Dropdown fields use alters with uitype: "code"

Dropdowns go in browserfields.alters, never a form section.

• Static: "alters": {"status": {"uitype": "code", "data": [{"value": "1", "text": "Active"}, {"value": "0", "text": "Inactive"}]}}
• Dynamic: "providerid": {"uitype": "code", "dataurl": "{{entire_url('../api/get_organizations.dspy')}}"}

Rules:

• dataurl must use {{entire_url('...')}} with a quoted string
• Endpoint must return a plain JSON array [{value, text}, ...] — no wrapping object; on error or empty data return []
• Prefer appcodes/appcodes_kv (model codes definitions) over inline data arrays for fixed options — see references/appcodes-pattern.md
• data_field is deprecated (old nested-response pattern {"data": {"organizations": [...]}}) — DO NOT use; wrapped formats make the dropdown silently not render
• valueField/textField are NOT deprecated — required when the data source returns keys other than value/text (see Pitfall 34)

Pitfall 4: Field hiding uses exclouded/editexclouded, not field-level "hidden": true

No {"name": "org_id", "widget": "hidden"}. Use "editexclouded": ["org_id"] (hides in edit form) and "browserfields": {"exclouded": ["org_id"]} (hides in list view).

Pitfall 5: Always load this skill BEFORE creating/modifying CRUD files

Never guess the CRUD format — the user has zero tolerance for guessed/improvised formats. If unsure about any property, load this skill and follow the examples verbatim.

Pitfall 6: Confidential fields should be hidden in browser view

Sensitive fields (API keys, passwords, tokens) must be in BOTH confidential_fields (server-side redaction) AND browserfields.exclouded (removed from the grid).

Pitfall 7: editable paragraph required for ALL CRUD files (even list-only views)

Every CRUD JSON file — including list-only views — MUST include editable with new_data_url, update_data_url, delete_data_url (optionally get_data_url). Without it the framework cannot process form submissions. URLs use {{entire_url('../api/xxx.dspy')}} format with ../ prefix:

"params": {"editable": {
    "new_data_url": "{{entire_url('../api/table_create.dspy')}}",
    "update_data_url": "{{entire_url('../api/table_update.dspy')}}",
    "delete_data_url": "{{entire_url('../api/table_delete.dspy')}}"
}}

It must be an object — never the string "default" (see Pitfall 41).

Pitfall 8: Field references MUST match model definitions exactly

All names in browserfields.exclouded, browserfields.alters keys, and editexclouded must be exact matches to fields in the table's model JSON (models/). Even a one-character difference fails. Common mismatches:

Wrong Field	Correct Field
org_id (not in model)	Remove it
sales_stage	current_stage
source	source_type
is_active	is_won_stage / is_lost_stage
changed_by	changed_by_id / changed_by_name

Pitfall 9: Non-CRUD files must not be placed in json/ directory

Files with custom structures ({"name": ..., "title": ..., "type": "page", "components": [...]}) are NOT CRUD definitions and cause framework failures. They belong as .ui files in wwwroot/.

Pitfall 10: Validate ALL json/ files when touching any file in the directory

When the task involves any CRUD file modification, scan EVERY .json file in json/ against this spec — not just the files directly involved. Zero tolerance for non-CRUD files left in or added to json/.

Pitfall 11: ID values must use appPublic.uniqueID.getID(), not uuid.uuid4()

Database id columns are typically VARCHAR(32); uuid.uuid4().replace('-', '') may exceed the column length. Always use:

from appPublic.uniqueID import getID
new_id = getID()

Applies to both .dspy API files and Python backend code.

Pitfall 12: Hand-written get_*_list.dspy SHADOWS framework auto-generated list endpoints

When json/{table}_list.json exists, the CRUD framework auto-generates the list endpoint. A hand-written wwwroot/api/get_{table}_list.dspy shadows it → 500 errors (filters on nonexistent fields), 403 errors (bypasses RBAC and logined_userorgid handling), silent data leakage (misses confidential_fields redaction). Rule: if json/{table}_list.json exists, do NOT create get_{table}_list.dspy. Only write hand-written dspy for custom business endpoints (create, update, delete, client-specific actions). Detect: list endpoint 500/403 → (1) does json/{alias}.json exist? (2) does wwwroot/api/get_{table}_list.dspy also exist? (3) if both → DELETE the hand-written dspy.

Pitfall 13: subtables[].subtable must reference an existing table with accessible UI

The subtable value needs: (1) a table definition in models/, (2) a corresponding wwwroot directory (or properly configured CRUD). A phantom reference (e.g. "subtable": "llmtype" with no model/wwwroot) breaks the UI silently — the tab renders but shows no data. CRITICAL — explicit url when default path lacks RBAC permissions: the framework's default path /module/subtable_name may not be registered in RBAC → subtab returns 401. Always set an explicit url to a .ui file that already exists:

"subtables": [{"field": "llmid", "title": "能力映射",
    "url": "{{entire_url('./llm_api_map_manage.ui')}}", "subtable": "llm_api_map"}]

Verify: find wwwroot -type d (CRUD dir for subtable), ls models/<subtable>.json, grep -r subtable_name wwwroot (existing .ui files). Default: always set url explicitly to a concrete .ui file rather than relying on framework path generation.

Pitfall 14: NEVER replace CRUD auto-generated endpoints with custom scripts

Auto-generated get_{table}.dspy handles RBAC, logined_userorgid, confidential_fields redaction, and DBFilter parsing correctly; custom scripts often violate .dspy conventions and create maintenance burden. Wrong: creating wwwroot/api/llm_list.dspy to replace get_llm.dspy just to add _text fields. Correct: keep the auto-generated endpoint and fix the dataurl API to return [{field_name, field_name_text}] instead. Only propose custom scripts for a genuine special requirement the framework cannot handle — and discuss the approach first. Validate configs: python references/validate_crud.py <module>/json/ --model-dir <module>/models/ (forbidden keys, missing required keys, nonexistent field refs, improper alters syntax). Module-wide audit: follow references/crud-audit-procedure.md.

Pitfall 15: NEVER manually add data_url to CRUD JSON

The framework auto-generates get_{table}.dspy; adding "data_url": "{{entire_url('../api/get_llm.dspy')}}" overrides it with a non-existent path → 500. Omit data_url entirely. Exception: only add data_url for a genuinely custom list endpoint whose .dspy file actually exists. For _text FK display, create get_search_{fieldname}.dspy (Pitfall 20) and reference it in alters[field].dataurl — do NOT modify the auto-generated list endpoint.

Pitfall 16: Generated CRUD directories are NOT committed to git

xls2ddl.xls2crud generates wwwroot/<table>/ (index.ui, get/add/update/delete .dspy) — these are auto-generated artifacts and must be gitignored (one .gitignore entry per table). The json/ CRUD definitions ARE committed (they are the source). See references/xls2crud-generation.md.

Pitfall 17: Custom data_url requires xls2ddl template support

For CRUD JSON data_url overrides to take effect, the xls2ddl template (data_browser_tmpl) must use {% if data_url %} (not {% if get_data_url %}). Fixed in xls2ddl commit 9f9a60a. If the generated index.ui ignores data_url, update xls2ddl.

Pitfall 18: Code-type (uitype: "code") field debugging — never blame bricks first

If a code-type field shows undefined, raw IDs, or wrong values, the problem is in the data layer (API response format, dataurl path, backend query) — NOT bricks (all code fields share the same component; a framework bug would hit all of them).

• Filter dropdown undefined → check dataurl returns [{value, text}] (ONLY correct format); check the path resolves; check the endpoint isn't swallowing errors into []
• Raw IDs in grid cells → the dataurl API must return [{value, text}] via SQL aliases:

orgs = await sor.sqlExe("select id as value, orgname as text from organization order by orgname", {})
return orgs

Do NOT edit the auto-generated wwwroot/<table>/get_<table>.dspy (regenerated from templates; it doesn't join reference tables). For grid _text resolution use the custom LEFT JOIN list endpoint — see Pitfall 47.

Pitfall 19: data_filter dropdown must have empty/default option — FIXED

Fixed in bricks commit f8f02c6 (get_filter_fields()): DataViewer auto-injects {value: '', text: ''} as the first option for all code-type filter fields (both data and dataurl sources), unless data already contains an empty/null/undefined entry. No backend changes needed. Old symptom (if unfixed): user selects a value and cannot reset to "show all".

Pitfall 20: codes fields need dedicated get_search_{fieldname}.dspy for filter dropdowns

When a model's codes section defines a foreign key (e.g. providerid → organization), the browserfields.alters dataurl should point to a dedicated get_search_{fieldname}.dspy (single-purpose, includes the "全部" fallback), not the generic list endpoint:

result = [{'value': '', 'text': '全部'}]
try:
    async with get_sor_context(request._run_ns, 'rbac') as sor:
        orgs = await sor.sqlExe("select id as value, orgname as text from organization order by orgname", {})
        return json.dumps([{'value': '', 'text': '全部'}] + list(orgs), ensure_ascii=False)
except Exception as e:
    debug(f'get_search_providerid error: {e}')
return json.dumps(result, ensure_ascii=False)

Key rules:

• Return [{value, text}] — SQL aliases id as value, name as text
• CRITICAL: on success prepend 全部 via [{'value': '', 'text': '全部'}] + list(orgs) — do NOT just return orgs (drops the 全部 option)
• On error: return fallback with only 全部; no imports allowed (json, get_sor_context, debug are pre-loaded)
• Name: get_search_{fieldname}.dspy in wwwroot/api/; register in load_path.py
• CRITICAL: also register in DB permission + rolepermission tables — load_path.py alone is NOT sufficient for new api/*.dspy endpoints. After deploying, run on the target server:

INSERT INTO permission (id, path) VALUES (REPLACE(UUID(),'-',''), '/module/api/endpoint.dspy');
INSERT INTO rolepermission (id, roleid, permid)
  SELECT REPLACE(UUID(),'-',''), 'logined', id FROM permission WHERE path='/module/api/endpoint.dspy';

Path MUST be /module/api/xxx.dspy (files in wwwroot/api/ are served at /module/api/). Without DB registration → 403 Forbidden even if load_path.py is correct. Use role 'logined' for endpoints any authenticated user can call. Architecture note: edit-form dropdowns use alters.dataurl; filter dropdowns use model codes → get_code.dspy. Both must be updated when changing a field's data source. See references/filter-vs-edit-dropdown.md.

Pitfall 21: NOT NULL DEFAULT columns MUST be in editexclouded

If a column has NOT NULL DEFAULT <value> (e.g. login_fail_count SMALLINT NOT NULL DEFAULT 0, created_at TIMESTAMP NOT NULL DEFAULT current_timestamp()) and users should NOT edit it, it MUST be in editexclouded. Otherwise the edit form renders an empty input and submits NULL → (1048, "Column 'xxx' cannot be null").

"editexclouded": ["id", "created_at", "login_fail_count", "last_login", "last_login_fail"]

Common culprits: DEFAULT current_timestamp() timestamps, DEFAULT 0 counters, DEFAULT '0' status columns.

Pitfall 22: record_toolbar pattern for state-change action buttons

Per-row action buttons (enable/disable, approve/reject, activate/deactivate):

"params": {"record_toolbar": [{
    "label": "启用", "actiontype": "dspy",
    "url": "/module/table/enable_record.dspy",
    "options": {"icon": "check", "cwidth": 16, "cheight": 9}
}]}

dspy handler pattern: check params_kw.get('id') (return {"widgettype":"Error",...} if missing); then db = DBPools(); async with db.sqlorContext(get_module_dbname('module_name')) as sor: await sor.U('table_name', {'id': params_kw.id, 'status_field': 'new_value'}) (dict with id + fields — do NOT pass a 3rd argument); return {"widgettype":"Message",...}. Key rules: each button needs its own .dspy in wwwroot/; dspy receives params_kw.id; register all paths in load_path.py; use cwidth/cheight in options (not fixed px). Global toolbar buttons (toolbar.tools + binds): see references/toolbar-tools-binds.md — sit above the list (unlike per-row record_toolbar), require selected_row: true + a binds entry with wid: "self"; covers urlwidget→PopupWindow, params_mapping, ${id}$ placeholder, DSPY return format, 403→load_path→restart flow.

Pitfall 23: dspy files must NOT have import statements — use pre-loaded modules only

Every .dspy file is injected into a pre-built async function context. Pre-loaded (NEVER import):

• From ahserver y_env: debug, exception, error, info, warning, critical; get_user, get_username, get_userorgid, get_userinfo; entire_url, i18n, redirect, clientinfo, terminalType
• Globals (injected at compile time): json, datetime (datetime/date/timedelta), time, DictObject, DBPools, get_sor_context, getID, curDateString, timestampstr, FileStorage, partial, params_kw, format_exc
• Only allowed import: from sqlor.filter import DBFilter Common violations (all WRONG): import json, import time, import datetime, from appPublic.uniqueID import getID, from appPublic.log import debug, from appPublic.dictObject import DictObject, from appPublic.timeUtils import curDateString, timestampstr, from sqlor.dbpools import get_sor_context, DBPools, from functools import partial, from ahserver.filestorage import FileStorage. If a module-internal function is needed (e.g. from llmage.utils import get_llmusage_by_id), export it via the module's load_XXX() in init.py (env.get_llmusage_by_id = get_llmusage_by_id), then dspy calls it directly. Audit before every commit touching .dspy: grep -rn '^import \|^from ' wwwroot/ --include='*.dspy' | grep -v 'sqlor.filter' — must return empty; delete any matching import line.

Pitfall 24: Tabular edit sends _text suffix fields — MUST strip before sor.U/sor.C

Tabular's edit form collects ALL field data including _text display columns (user_status_text, orgid_text, sync_from_text) that are NOT real DB columns; passing them to sor.U/sor.C fails silently or skips the update. In EVERY add/update .dspy after ns = params_kw.copy():

ns = params_kw.copy()
for k, v in ns.items():
    if v == 'NaN' or v == 'null':
        ns[k] = None
# remove _text suffix fields sent by Tabular (not real DB columns)
for k in list(ns.keys()):
    if k.endswith('_text'):
        ns.pop(k, None)

Affects ALL add_*.dspy/update_*.dspy generated by xls2crud, plus any custom form handler receiving Tabular/Form code-type fields. Detection: grep for _text in params_kw before sor.C/U calls.

Pitfall 25: Delegate-pattern endpoints (return result from helper) are valid — do NOT flag as format errors

A _create.dspy / _update.dspy / _delete.dspy that just calls a helper and returns its result (result = await create_marketing(request, params_kw); return result) is VALID — the helper (in init.py) returns the correct widgettype format. Do NOT flag as no_widgettype errors. Only flag endpoints that construct their own return value (e.g. {'success': True, ...}) without widgettype. Detect: last line return result / return json.loads(result) + a result = await helper(...) line → delegate pattern, skip format checking.

Pitfall 26: Model name variants (dot vs hyphen) must be covered in model_mappings

LLM API responses may use dotted versions (doubao-seedance-2.0) while pricing YAML entries use hyphens (doubao-seedance-2-0); model_mappings that only map full version suffixes miss the dot-variants → {config_data=..., mismatched} / 没有找到合适的定价. Fix — map all dot-variants explicitly:

model_mappings:
  doubao-seedance-2.0: doubao-seedance-2-0
  doubao-seedance-2.0-fast: doubao-seedance-2-0-fast
  doubao-seedance-2-0-260128: doubao-seedance-2-0

Detection: compare the incoming model field against pricing filter values; if they differ only by . vs - in version numbers, add the mapping. Cross-skill: also affects pricing-data-format.

Pitfall 27: Tabular sends request params via data_params, NOT params

bricks DataViewer reads this.opts.data_params as the default request parameters — NOT this.opts.params (which is why a Tabular gets no discountid and returns empty data). Wrong: "params": {"discountid": "{{params_kw.discountid}}"}. Correct:

"data_url": "...dspy",
"data_params": {"discountid": "{{params_kw.discountid}}"}

Applies to ALL hand-written Tabular widgets, including subtable pages; CRUD auto-generated files handle this correctly via the template.

Pitfall 28: data_filter conflicts with logined_userorgid — FIXED in xls2ddl

Fixed in xls2ddl commit ebd4b4a: the generated get_<table>.dspy now injects logined_userorgid/logined_userid conditions into filterjson before DBFilter processes it (appending {'field': '<orgfield>', 'op': '=', 'var': '__logined_orgid__'} + ns['__logined_orgid__'] = userorgid; same for userid). Ensures both work together. Old workaround (manual ownerid filter in dspy / const in data_filter) no longer needed. After updating xls2ddl, regenerate affected modules.

Pitfall 29: Subtables auto-population — use field + mapping to pass parent values

For a subtable's add form to receive the parent record's ID (e.g. supplier contract needs supplier_id): (1) add the field to the subtable's editexclouded so users don't edit it; (2) carry the value via new_data_url:

"editexclouded": ["id", "resellerid", "supplier_id"],
"editable": {"new_data_url": "{{entire_url('../api/create.dspy')}}?supplier_id={{params_kw.get('supplier_id','')}}"}

The xls2crud params_mapping.mapping sends parent.id → subtable.field to the subtable page as URL params; the add form's new_data_url carries it to the dspy.

Pitfall 30: xls2ddl json.dumps(true) generates Python-invalid true — MUST wrap in json.loads()

Generated dspy fails NameError: name 'true' is not defined (JSON booleans like "not_null": true are valid JSON but not Python). Fix in xls2ddl tmpls.py — CRITICAL: the json.loads() call MUST have single quotes around the Jinja2 expression (without quotes it receives a dict → TypeError: the JSON object must be str, bytes or bytearray, not dict):

# WRONG:  tblfields = {{json.dumps(fields, ensure_ascii=False)}}
# WRONG:  tblfields = json.loads({{json.dumps(fields, ensure_ascii=False)}})   # dict, not string
# CORRECT:
tblfields = json.loads('{{json.dumps(fields, ensure_ascii=False)}}')
filterjson = json.loads('{{json.dumps(data_filter, ensure_ascii=False)}}')
ns['sort'] = json.loads('{{json.dumps(sortby)}}')

Apply ONLY to Python-context lines (tblfields, filterjson, sort arrays). JSON-context lines (browserfields, toolbar, binds) use true/false correctly and must NOT be wrapped. Commit: xls2ddl af6006f.

Pitfall 31: Post-insert re-query uses primary, not pkey — xls2ddl template bug

OperationalError: (1054, "Unknown column 'None' in 'WHERE'") with SELECT * FROM <table> WHERE None = %s after adding a record (xls2ddl fc91486 added a post-insert re-query ... WHERE {{summary[0].pkey}} = ${id}$; model JSONs use primary (array), not pkey → Jinja2 renders None). Fix: {{summary[0].pkey}} → {{summary[0].primary[0]}} in xls2ddl tmpls.py, then regenerate all modules.

Pitfall 32: logined_userorgid generates WHERE None = %s — FIXED in xls2ddl

Same (1054, "Unknown column 'None' in 'WHERE'") symptom when logined_userorgid is configured but the user's orgid is not set. Fix: update xls2ddl to commit ebd4b4a or later, then regenerate ALL affected modules:

cd ~/repos/xls2ddl && git pull
cd ~/repos/<module> && PYTHONPATH=~/repos/xls2ddl python3 -m xls2ddl.xls2crud -m models -o wwwroot <mod> json/*.json

Pitfall 33: valueField/textField in alters MUST also have uitype: "code"

Without uitype: "code" the framework treats the field as plain text and silently discards the textField mapping — grid shows raw IDs even though the API returns {fieldname}_text columns:

// WRONG: {"llmid": {"valueField": "llmid", "textField": "llmid_text"}}
// CORRECT:
{"llmid": {"uitype": "code", "valueField": "llmid", "textField": "llmid_text"}}

Applies to both hand-written .ui files and CRUD JSON browserfields.alters. See Pitfall 34 for the full valueField/textField pattern.

Pitfall 34: valueField/textField in alters apply to BOTH filter form AND edit form

browserfields.alters is a single configuration shared by the list grid, the filter/search form, and the add/edit form. Use valueField/textField when: the get_search_*.dspy returns {fieldname, fieldname_text} keys (not {value, text}), the list endpoint returns {fieldname}_text columns, and you need the stored value key to be the actual field name:

"providerid": {
    "uitype": "code",
    "dataurl": "{{entire_url('../api/get_search_providerid.dspy')}}",
    "valueField": "providerid",
    "textField": "providerid_text"
}

The dspy MUST return those exact keys: select id as providerid, orgname as providerid_text from organization order by orgname, and the "全部" fallback must use the same keys ({'providerid': '', 'providerid_text': '全部'}). Key rules: valueField/textField must exactly match the endpoint's returned keys; the same pair is used in filter form AND edit form (they cannot differ); if the dspy returns {value, text}, do NOT set valueField/textField (framework defaults). Symptom if wrong: filter dropdown shows undefined/raw IDs; edit form sends wrong values; filter selection doesn't match stored data.

Pitfall 35: MUST regenerate index.ui on each server after modifying CRUD JSON alters

wwwroot/<table>/ (index.ui) is auto-generated by xls2crud and gitignored (Pitfall 16) — git pull on a server leaves stale UI. After ANY change to browserfields.alters (dataurl/valueField/textField/uitype/data), browserfields.exclouded, editexclouded, data_filter, filter_labels, filter_title, subtables, record_toolbar, editor.binds, or model definitions in models/, re-run on EACH environment:

cd /path/to/module
PYTHONPATH=/path/to/xls2ddl python3 -m xls2ddl.xls2crud -m models -o wwwroot <modulename> json/<table>.json

Pitfall 36: CRUD page not triggering data request — diagnose CRUD spec FIRST, not permissions

When a CRUD list page loads but does NOT trigger the get_<table>.dspy request, the root cause is the CRUD spec or generated template — NOT RBAC/permissions. Diagnostic order: (1) generated index.ui has a data_url pointing to the right endpoint; (2) CRUD JSON has required structure (tblname, params, editable); (3) get_<table>.dspy exists in the CRUD directory; (4) diff against a working module (e.g. llmage/llm, pricing); (5) only then check RBAC/load_path.py. Common causes (frequency order): stale index.ui (not regenerated), data_url missing/incorrect in generated index.ui, CRUD JSON missing required keys, outdated xls2ddl template. Anti-pattern: assume permissions first, shuffle PATHS_ANY/PATHS_LOGINED, or blame bricks before checking the CRUD config.

Pitfall 37: data_url vs get_data_url in generated index.ui — two different mechanisms

1. data_url (outside editable block) — initial page load by Tabular/DataViewer. Defaults to ./get_<table>.dspy; overridden if CRUD JSON has a data_url key.
2. get_data_url (inside editable block, optional) — overrides the data URL with extra parameters (e.g. ?pagerows=50). Only generated when the CRUD JSON explicitly defines it:

"editable": {"get_data_url": "{{entire_url('get_llmusage.dspy')}}?pagerows=50", "new_data_url": "...", ...}

Most CRUD pages only need the auto data_url. Tabular uses data_url on initial render, then get_data_url for subsequent refreshes if defined. If neither exists in generated index.ui, the table renders empty with no network request.

Pitfall 38: data_filter MUST use DBFilter tree structure, NOT {"fields": [...]}

A flat "fields": [{"field": ..., "title": ..., "uitype": "code"}] silently fails (search does nothing / popup doesn't appear). Correct — DBFilter tree with op/var:

"data_filter": {"AND": [
    {"field": "supplier_org_id", "op": "=", "var": "supplier_org_id"},
    {"field": "resource_type", "op": "=", "var": "resource_type"}
]}

Add a "filter_labels" object for Chinese labels.

Pitfall 39: NEVER directly edit auto-generated .ui files — always modify source configs

Files in wwwroot/<table>/ (especially index.ui) are auto-generated by xls2crud — never edit directly, even in production (e.g. no sed -i on production index.ui). Correct approach: (1) find the source config — json/<table>.json for CRUD-generated UIs, wwwroot/<custom_name>.ui for hand-written UIs; (2) modify the source; (3) regenerate (CRUD only, see Pitfall 35 command); (4) deploy. How to tell if auto-generated: json/<table>.json exists → yes; directory contains index.ui + get_<table>.dspy → yes; files outside wwwroot/<table>/ (e.g. wwwroot/custom_feature.ui) → hand-written, editable directly.

Pitfall 40: CRUD list performance — FOUR critical optimizations required

Symptom: list page 3+ seconds. File location: never edit the gitignored, regenerated wwwroot/<table>/get_<table>.dspy — write wwwroot/api/<table>_list.dspy and point editable.get_data_url at it (committed, survives regeneration). Apply optimizations in order (each 2-5x; together 20-40x):

1. sqlPaging wraps queries in subqueries (slowest) → separate count + data queries: select count(*) as cnt ... then select col1, col2 ... limit {rows_per_page} offset {(page-1)*rows_per_page} with page = int(ns.get('page', 1)), rows_per_page = int(ns.get('rows', ns.get('pagerows', 50)))
2. default_filterjson generates LIKE for ALL fields incl. large TEXT (full table scan) → exclude TEXT columns: filter_fields = [f['name'] for f in ori_fields if f['name'] not in ('usages', 'ioinfo')]
3. SELECT * pulls large TEXT/BLOB (off-page storage I/O) → explicit column list
4. DBFilter + ArgsConvert framework overhead → bypass entirely with raw SQL + manual WHERE on 5-7 common filter fields (conditions = ['1=1'], append 'field=${var}$' + ns['var'] = value only when value present), using DBPools() + get_module_dbname('module_name'); return {'success': True, 'total': total, 'rows': rows, 'page': page, 'page_size': rows_per_page}; wrap in try/except with debug(format_exc()) and {'success': False, ...} fallback. Real-world (llmage/llmusage, 17 columns): sqlPaging + all-fields filter + SELECT * → 8-12s; after #1: 3-4s; after #2: 1-2s; after #3: 0.8-1.5s; after #4: 0.2-0.5s.

Pitfall 41: "editable": "default" string causes xls2ui serialization error

"editable": "default" (string instead of object) → build.sh/xls2ui fails with TypeError: Object of type builtin_function_or_method is not JSON serializable at json.dumps(binds, ...). Fix: replace with the full object containing get_data_url, new_data_url, update_data_url, delete_data_url (the "default" shorthand is unsupported). This is a case of Pitfall 7 focused on format — must be an object, not a string.

Pitfall 42: Field title renaming — modify models/<table>.json, NOT CRUD/UI files

To rename display titles (e.g. "用户id" → "username"), change fields[].title in models/<table>.json. The model title is the single source of truth, propagated to list grid headers, filter form labels, edit form labels, and add form labels. Do NOT edit titles in json/<table>.json browserfields (overrides only), generated wwwroot/<table>/index.ui (build artifact), or wwwroot/api/get_<table>.dspy (titles aren't in the query layer). After renaming, regenerate CRUD files. Real example (llmage/llmusage 2026-06-25, commit 13c123c): userorgid "用户机构"→"orgname", ownerid "模型机构"→"orgname", userid "用户id"→"username", llmid "模型id"→"model".

Pitfall 43: logined_userorgid and logined_userid go in params, NOT browserfields

Putting them inside browserfields crashes xls2ui with JSONDecodeError or places them wrongly in generated files. They are first-class params keys, siblings of sortby, data_filter, editable:

"params": {
    "logined_userorgid": "customerid",
    "browserfields": {"exclouded": ["id"]}
}

Pitfall 44: editable.new_data_url was NEVER read by xls2ddl template — fixed in fb613d0

CRUD JSON with params.editable.new_data_url set, but the generated index.ui still uses the default add_<table>.dspy (custom create dspy never called). Root cause: the template checked {% if new_data_url %} at top level, but after desc.update(crud_data.params.copy()) the key is nested in desc.editable; Jinja2 resolves it to None → {% else %} always runs. Fix: xls2ddl commit fb613d0+ — template checks {% if (editable and editable.new_data_url) or new_data_url %} (same for delete_data_url, update_data_url). Was masked while pre-gitignore generated dspy files existed on servers. Regenerate after updating xls2ddl.

Pitfall 45: filter_fields generated without inline data from browserfields.alters

Search popup crashes with TypeError: Cannot read properties of undefined (reading 'length') at bricks.UiCode.build_options when a data_filter field has inline data only in browserfields.alters (e.g. {"uitype":"code","data":[...]}) but no corresponding model codes entry — filter_fields gets a code header with data: undefined. Fix: xls2ddl commit faa571f+ (merges alters data/dataurl/valueField/textField into filter_fields, applies alters uitype override). Regenerate affected CRUD pages.

Pitfall 46: CRUD JSON files must be PURE JSON — NO Jinja2 control-flow blocks

json/ files are parsed as pure JSON by xls2ddl.xls2crud; {% if %}, {% for %}, {% endif %} cause JSONDecodeError at parse time. Use fixed values instead (e.g. hardcode "width": "40%" instead of {% if params_kw._is_mobile %}). {{entire_url('...')}} placeholders inside JSON string values ARE acceptable — handled during URL generation. This is distinct from .ui template files which DO support Jinja2. See references/crud-json-rules.md.

Pitfall 47: Custom list endpoint for code resolution — LEFT JOIN reference tables for _text grid display

Symptom: form/filter dropdowns show names (via uitype: "code" + dataurl) but the list grid still shows raw IDs. Root cause: the auto-generated get_<table>.dspy queries only the base table — no reference-table joins. uitype:"code" + dataurl only controls form/filter dropdowns, NOT grid cells; inline data arrays for status/type fields also do NOT auto-resolve for grid cells. Pitfalls 14/15 don't cover this — the auto-generated list never LEFT JOINs, and a custom list endpoint is the only way to get grid code resolution. Approach A — COALESCE (recommended, the only reliable one): replace raw IDs with display names directly in SQL output — guarantees grid rendering regardless of bricks _text detection:

sql = '''select a.id, a.domain,
    COALESCE(b.orgname, a.resellerid) as resellerid,
    COALESCE(c.orgname, a.orgid) as orgid,
    CASE a.status WHEN 'active' THEN '启用' WHEN 'inactive' THEN '停用' ELSE a.status END as status
from (select * from tenant_domain where 1=1 [[filterstr]]) a
left join (select id, orgname from organization) b on a.resellerid = b.id
left join (select id, orgname from organization) c on a.orgid = c.id'''

Filter WHERE applies on the inner query with raw IDs before COALESCE. Con: the edit form must use a separate _get.dspy returning raw IDs. Approach B — _text suffix: return resellerid_text/orgid_text columns via LEFT JOIN; bricks Tabular MAY auto-detect _text columns but detection is unreliable (2026-07-10 testing: did NOT render) — fall back to Approach A. Placement (two valid locations): (1) wwwroot/{alias}/get_{alias}.dspy — CRUD framework auto-discovers; no get_data_url needed; directory may be gitignored (use git add -f); takes precedence over get_data_url; (2) wwwroot/api/<table>_list.dspy + editable.get_data_url — requires xls2ddl ≥ fb613d0 (Pitfall 44). Key rules: LEFT JOIN alias MUST match the field name; never edit the gitignored wwwroot/<table>/get_<table>.dspy; handle status/type fields in SQL with CASE WHEN; new dspy endpoints need BOTH permission + rolepermission (role 'logined') SQL rows (see Pitfall 20). CRITICAL deployment pitfalls (2026-07-10):

1. Server restart required for CRUD directory dspys: custom get_{alias}.dspy files in CRUD directories are cached at server STARTUP — hot-reload applies to wwwroot/api/ but NOT to auto-generated CRUD directories. After deploying, restart Sage (./stop.sh && ./start.sh).
2. default_filterjson trap — ns field pollution: any field set in ns before default_filterjson(fields, ns) becomes an implicit filter. NEVER set business fields in ns — only framework vars (__logined_orgid__, userorgid). Example: ns['resellerid'] = userorgid leaked into default_filterjson and filtered the list to 1 row.
3. Approach A (COALESCE) is the only reliable option — Approach B's _text columns did not render in practice.