pipeline_core/skills_library/all/reference-module-protection/SKILL.md

name	category	description	trigger
reference-module-protection	devops	Rules and workflows for protecting foundation/reference modules from accidental modification	When asked to fix issues that might involve modifying foundation/reference modules (apppublic, ahserver, sqlor, appbase, rbac)

Reference Module Protection Rules

Reference Modules (READ-ONLY)

The following modules are foundation modules and must NEVER be modified:

1. apppublic - Python utility module (Singleton, log, jsonConfig, etc.)
2. ahserver - Web server framework (aiohttp-based)
3. sqlor - Database module (SQL ORM/execution)
4. appbase - Base functionality module (code management, parameter management)
5. rbac - RBAC authentication module (user management, permission control)

These live under ~/repos/ and are installed via pip into application projects.

Git Recovery Workflow

When reference modules lose their .git directories:

1. Clone from remote: git clone git@git.opencomputing.cn:yumoqing/<module>.git /tmp/<module>_remote
2. Copy .git to local: cp -a /tmp/<module>_remote/.git ~/repos/<module>/.git
3. Stage only actual source changes (exclude __pycache__, .egg-info, build/)
4. Commit and push local modifications
5. Verify with git status --short
6. Clean up: rm -rf /tmp/<module>_remote

RBAC Module Has Built-in Login/Register UI (DO NOT RECREATE)

The rbac module already provides complete user authentication UI under wwwroot/user/:

File	Purpose
login.ui	Login dialog (PopupWindow with TabPanel for login + register tabs)
register.ui / register.dspy	User registration form and logic
up_login.dspy	Login credential verification
logout.dspy	Session logout
myrole.ui	Display current user's roles
userinfo.ui	User profile information
reset_password/	Password reset flow
user.ui / user_panel.ui	User management panel

When developing new modules that need authentication, route users to rbac's existing login (/user/login.ui) instead of creating custom login pages in your module's wwwroot.

RBAC Module Critical Internal Behavior (DO NOT MODIFY rbac)

The rbac module's userperm.py has hardcoded role ID checks for convention roles:

if r.id == 'anonymous': k = 'anonymous'
elif r.id == 'any': k = 'any'
elif r.id == 'logined': k = 'logined'

This means convention roles MUST use these exact fixed IDs — do NOT use getID() or random UUIDs when creating 'any', 'logined', or 'anonymous' roles. If you accidentally created them with random IDs, you must fix the role IDs to match these hardcoded strings.

Application-Layer Workaround Pattern for RBAC Limitations

When rbac has limitations (e.g., missing PUBLIC_PATHS support), work around it at the application layer:

1. Create app/perm_config.py to define permission matrix (roles, perms per path pattern)
2. Create app/init_permissions.py to:
   • Scan module wwwroot for .ui/.dspy files
   • Register CRUD paths (/{modulename}/{tablename}/)
   • Create roles with correct fixed IDs for convention roles
   • Map role-permission relationships
   • Run automatically at application startup

Example role ID setup:

# Convention roles MUST use fixed IDs (rbac userperm.py checks these)
for role_id in ['any', 'logined', 'anonymous']:
    # Use fixed ID, NOT getID() random UUID
    role = roleType(id=role_id, code=role_id, name=role_id)
    sorInsert(role)

Known Reference Module Bugs (MUST be patched for any new module)

The following bugs have been identified in reference modules. They must be patched for any new application to function — they cannot be worked around at the application layer.

Bug 1: rbac/rbac/userperm.py - check_roles_path() lacks wildcard support

Problem: The method only does exact string matching (if path in paths). Permission patterns like /customer_management/** in perm_config.py never work. Also doesn't handle /main URL prefix (URL /main/xxx vs stored permission /xxx).

Solution: DO NOT modify rbac. Instead, expand wildcards at init time in the application layer.

The correct approach is to expand ** patterns during permission initialization by scanning the actual filesystem:

1. init_permissions.py scans wwwroot/ for all .ui and .dspy files
2. For each ** pattern in perm_config.py, match it against actual file paths
3. Register the expanded exact paths to the database
4. Also register the /main/ prefixed version (since URLs come in with /main prefix)

def expand_wildcard(pattern: str, all_paths: set) -> set:
    """Expand a ** wildcard pattern into exact paths."""
    if '**' not in pattern:
        return {pattern}
    prefix = pattern.replace('**', '').rstrip('/')
    return {p for p in all_paths if p.startswith(prefix)}

# During init:
all_files = set()
for root, dirs, files in os.walk('wwwroot'):
    for f in files:
        if f.endswith(('.ui', '.dspy')):
            rel = os.path.relpath(os.path.join(root, f), 'wwwroot')
            all_files.add('/' + rel)

for path_pattern, roles in role_paths.items():
    expanded = expand_wildcard(path_pattern, all_files)
    for exact_path in expanded:
        # Register both /xxx and /main/xxx variants
        register_permission(sor, exact_path, roles)
        register_permission(sor, '/main' + exact_path, roles)

This way rbac stays untouched — all ** expansion happens at init time, and the DB contains exact paths that rbac's string matching can handle.

OLD approach (DO NOT USE): Patching rbac's check_roles_path() to add wildcard matching. This violates the reference module protection rule and causes upgrade/maintenance issues.

Bug 2: ahserver/ahserver/url2file.py - url2ospath() doesn't strip URL prefix

Problem: URL /main/rbac/user/up_login.dspy resolves to wwwroot/main/rbac/... (wrong) instead of wwwroot/rbac/.... The URL prefix (e.g., /main) must be stripped before filesystem path resolution.

Fix: In url2ospath(), add prefix stripping:

def url2ospath(self, url: str) -> str:
    url = url.split('?')[0]
    if len(url) > 0 and url[-1] == '/':
        url = url[:-1]
    # Strip URL prefix before resolving to filesystem
    if self.starts and url.startswith(self.starts):
        url = url[len(self.starts):]
        if not url.startswith('/'):
            url = '/' + url
    # ... rest of method unchanged

Bug 3: ahserver/ahserver/processorResource.py - path normalization in _handle()

Problem: Line 353: self.request_filename = self.url2file(str(request.path)) should be self.url2file(self.url2path(str(request.path))). Without url2path(), the /main prefix is not stripped.

Fix: Change line 353 to:

self.request_filename = self.url2file(self.url2path(str(request.path)))

CRUD Auto-Generated Directories — NEVER commit to git

CRUD auto-generated directories under wwwroot/ (table-named subdirectories containing add_*/get_*/update_*/delete_*.dspy + index.ui) are generated by xls2ui and must NEVER be committed to git repos. They are environment-specific and create merge conflicts on git pull.

Before every commit, check: git status — if any table-named directories appear under wwwroot/, remove them:

git rm -r wwwroot/organization wwwroot/role wwwroot/users ...  2>/dev/null

After git pull conflicts: if production has untracked CRUD files that conflict with incoming tracked versions:

git checkout -- wwwroot/<dir>/   # discard local
# or
rm -rf wwwroot/<dir>/            # nuke and re-gen
git pull

Prevention: add CRUD directories to .gitignore immediately when a new module is scaffolded. Files modified: 54 deletions covering organization, orgtypes, permission, provider, reseller, role, rolepermission, userapp, userdepartment, userrole, users.

Additional Pitfalls

wwwroot has no index.html -- /main/ returns 500

The wwwroot/ directory does NOT contain an index.html file. When users navigate to https://host/main/ or https://host/, the backend cannot find a default file and throws 'NoneType' object is not iterable. Fix: Configure nginx to redirect /main/ to /main/base.ui (the app's entry point):

location = /main/ { return 302 /main/base.ui; }
location = / { return 302 /main/; }

Permission init requires app restart

The app loads permissions into memory at startup. Running init_permissions.py updates the DB, but permissions won't take effect until the app is restarted (pkill -f integrated_crm_app.py then restart).

.dspy files use Python syntax, NOT JavaScript

• true / false must be True / False (capital T/F)
• Common in generated code from JSON model definitions where nullable: true is copied directly into Python code
• Fix: sed -i 's/: true$/: True/; s/: true,/: True,/; s/: false$/: False/; s/: false,/: False,/' *.dspy

Permission path mismatch

• perm_config.py defines paths like /customer_management/** (no /main prefix)
• init_permissions.py registers these exact paths to the DB
• URL requests come in as /main/customer_management/...
• The rbac check_roles_path() must handle this prefix normalization (see Bug 1)

Convention role anonymous needs login page permissions

• The anonymous role (unauthenticated users) must have explicit permissions for login/registration pages
• Without this, even the login page returns 401
• Required anonymous permissions: /main/login.ui, /main/login.dspy, /rbac/user/up_login.dspy, /rbac/user/register.ui, /rbac/user/login.ui

Verification Checklist

Before any git commit involving reference modules:

for mod in apppublic ahserver sqlor appbase rbac; do
  cd ~/repos/$mod && git status --short
done

All reference modules should show clean status (no changes).