---
name: reference-module-protection
category: devops
description: Rules and workflows for protecting foundation/reference modules from accidental modification
trigger: When asked to fix issues that might involve modifying foundation/reference modules (apppublic, ahserver, sqlor, appbase, rbac)
---

# Reference Module Protection Rules

## Reference Modules (READ-ONLY)

The following modules are **foundation modules** and must NEVER be modified:

1. `apppublic` - Python utility module (Singleton, log, jsonConfig, etc.)
2. `ahserver` - Web server framework (aiohttp-based)
3. `sqlor` - Database module (SQL ORM/execution)
4. `appbase` - Base functionality module (code management, parameter management)
5. `rbac` - RBAC authentication module (user management, permission control)

These live under `~/repos/` and are installed via pip into application projects.

## Git Recovery Workflow

When reference modules lose their `.git` directories:

1. Clone from remote: `git clone git@git.opencomputing.cn:yumoqing/<module>.git /tmp/<module>_remote`
2. Copy `.git` to local: `cp -a /tmp/<module>_remote/.git ~/repos/<module>/.git`
3. Stage only actual source changes (exclude `__pycache__`, `.egg-info`, `build/`)
4. Commit and push local modifications
5. Verify with `git status --short`
6. Clean up: `rm -rf /tmp/<module>_remote`

## RBAC Module Has Built-in Login/Register UI (DO NOT RECREATE)

The rbac module already provides complete user authentication UI under `wwwroot/user/`:

| File | Purpose |
|------|---------|
| `login.ui` | Login dialog (PopupWindow with TabPanel for login + register tabs) |
| `register.ui` / `register.dspy` | User registration form and logic |
| `up_login.dspy` | Login credential verification |
| `logout.dspy` | Session logout |
| `myrole.ui` | Display current user's roles |
| `userinfo.ui` | User profile information |
| `reset_password/` | Password reset flow |
| `user.ui` / `user_panel.ui` | User management panel |

When developing new modules that need authentication, route users to rbac's existing login (`/user/login.ui`) instead of creating custom login pages in your module's wwwroot.

## RBAC Module Critical Internal Behavior (DO NOT MODIFY rbac)

The rbac module's `userperm.py` has **hardcoded role ID checks** for convention roles:
```python
if r.id == 'anonymous': k = 'anonymous'
elif r.id == 'any': k = 'any'
elif r.id == 'logined': k = 'logined'
```
This means convention roles MUST use these exact fixed IDs — do NOT use `getID()` or random UUIDs when creating 'any', 'logined', or 'anonymous' roles. If you accidentally created them with random IDs, you must fix the role IDs to match these hardcoded strings.

## Application-Layer Workaround Pattern for RBAC Limitations

When rbac has limitations (e.g., missing PUBLIC_PATHS support), work around it at the application layer:

1. Create `app/perm_config.py` to define permission matrix (roles, perms per path pattern)
2. Create `app/init_permissions.py` to:
   - Scan module wwwroot for .ui/.dspy files
   - Register CRUD paths (/{modulename}/{tablename}/)
   - Create roles with correct fixed IDs for convention roles
   - Map role-permission relationships
   - Run automatically at application startup

Example role ID setup:
```python
# Convention roles MUST use fixed IDs (rbac userperm.py checks these)
for role_id in ['any', 'logined', 'anonymous']:
    # Use fixed ID, NOT getID() random UUID
    role = roleType(id=role_id, code=role_id, name=role_id)
    sorInsert(role)
```

## Known Reference Module Bugs (MUST be patched for any new module)

The following bugs have been identified in reference modules. They **must be patched** for any new application to function — they cannot be worked around at the application layer.

### Bug 1: rbac/rbac/userperm.py - `check_roles_path()` lacks wildcard support

**Problem:** The method only does exact string matching (`if path in paths`). Permission patterns like `/customer_management/**` in `perm_config.py` never work. Also doesn't handle `/main` URL prefix (URL `/main/xxx` vs stored permission `/xxx`).

**Solution: DO NOT modify rbac. Instead, expand wildcards at init time in the application layer.**

The correct approach is to expand `**` patterns during permission initialization by scanning the actual filesystem:

1. `init_permissions.py` scans `wwwroot/` for all `.ui` and `.dspy` files
2. For each `**` pattern in `perm_config.py`, match it against actual file paths
3. Register the expanded exact paths to the database
4. Also register the `/main/` prefixed version (since URLs come in with `/main` prefix)

```python
def expand_wildcard(pattern: str, all_paths: set) -> set:
    """Expand a ** wildcard pattern into exact paths."""
    if '**' not in pattern:
        return {pattern}
    prefix = pattern.replace('**', '').rstrip('/')
    return {p for p in all_paths if p.startswith(prefix)}

# During init:
all_files = set()
for root, dirs, files in os.walk('wwwroot'):
    for f in files:
        if f.endswith(('.ui', '.dspy')):
            rel = os.path.relpath(os.path.join(root, f), 'wwwroot')
            all_files.add('/' + rel)

for path_pattern, roles in role_paths.items():
    expanded = expand_wildcard(path_pattern, all_files)
    for exact_path in expanded:
        # Register both /xxx and /main/xxx variants
        register_permission(sor, exact_path, roles)
        register_permission(sor, '/main' + exact_path, roles)
```

This way rbac stays untouched — all `**` expansion happens at init time, and the DB contains exact paths that rbac's string matching can handle.

**OLD approach (DO NOT USE):** Patching rbac's `check_roles_path()` to add wildcard matching. This violates the reference module protection rule and causes upgrade/maintenance issues.

### Bug 2: ahserver/ahserver/url2file.py - `url2ospath()` doesn't strip URL prefix

**Problem:** URL `/main/rbac/user/up_login.dspy` resolves to `wwwroot/main/rbac/...` (wrong) instead of `wwwroot/rbac/...`. The URL prefix (e.g., `/main`) must be stripped before filesystem path resolution.

**Fix:** In `url2ospath()`, add prefix stripping:
```python
def url2ospath(self, url: str) -> str:
    url = url.split('?')[0]
    if len(url) > 0 and url[-1] == '/':
        url = url[:-1]
    # Strip URL prefix before resolving to filesystem
    if self.starts and url.startswith(self.starts):
        url = url[len(self.starts):]
        if not url.startswith('/'):
            url = '/' + url
    # ... rest of method unchanged
```

### Bug 3: ahserver/ahserver/processorResource.py - path normalization in `_handle()`

**Problem:** Line 353: `self.request_filename = self.url2file(str(request.path))` should be `self.url2file(self.url2path(str(request.path)))`. Without `url2path()`, the `/main` prefix is not stripped.

**Fix:** Change line 353 to:
```python
self.request_filename = self.url2file(self.url2path(str(request.path)))
```

## CRUD Auto-Generated Directories — NEVER commit to git

CRUD auto-generated directories under `wwwroot/` (table-named subdirectories containing `add_*/get_*/update_*/delete_*.dspy` + `index.ui`) are generated by `xls2ui` and must NEVER be committed to git repos. They are environment-specific and create merge conflicts on `git pull`.

**Before every commit, check:** `git status` — if any table-named directories appear under `wwwroot/`, remove them:
```bash
git rm -r wwwroot/organization wwwroot/role wwwroot/users ...  2>/dev/null
```

**After git pull conflicts:** if production has untracked CRUD files that conflict with incoming tracked versions:
```bash
git checkout -- wwwroot/<dir>/   # discard local
# or
rm -rf wwwroot/<dir>/            # nuke and re-gen
git pull
```

**Prevention:** add CRUD directories to `.gitignore` immediately when a new module is scaffolded. Files modified: 54 deletions covering organization, orgtypes, permission, provider, reseller, role, rolepermission, userapp, userdepartment, userrole, users.

## Additional Pitfalls

### wwwroot has no index.html -- /main/ returns 500
The `wwwroot/` directory does NOT contain an `index.html` file. When users navigate to `https://host/main/` or `https://host/`, the backend cannot find a default file and throws `'NoneType' object is not iterable`. **Fix**: Configure nginx to redirect `/main/` to `/main/base.ui` (the app's entry point):
```nginx
location = /main/ { return 302 /main/base.ui; }
location = / { return 302 /main/; }
```

### Permission init requires app restart
The app loads permissions into memory at startup. Running `init_permissions.py` updates the DB, but permissions won't take effect until the app is restarted (`pkill -f integrated_crm_app.py` then restart).

### .dspy files use Python syntax, NOT JavaScript
- `true` / `false` must be `True` / `False` (capital T/F)
- Common in generated code from JSON model definitions where `nullable: true` is copied directly into Python code
- Fix: `sed -i 's/: true$/: True/; s/: true,/: True,/; s/: false$/: False/; s/: false,/: False,/' *.dspy`

### Permission path mismatch
- `perm_config.py` defines paths like `/customer_management/**` (no `/main` prefix)
- `init_permissions.py` registers these exact paths to the DB
- URL requests come in as `/main/customer_management/...`
- The rbac `check_roles_path()` must handle this prefix normalization (see Bug 1)

### Convention role `anonymous` needs login page permissions
- The `anonymous` role (unauthenticated users) must have explicit permissions for login/registration pages
- Without this, even the login page returns 401
- Required anonymous permissions: `/main/login.ui`, `/main/login.dspy`, `/rbac/user/up_login.dspy`, `/rbac/user/register.ui`, `/rbac/user/login.ui`

## Verification Checklist

Before any git commit involving reference modules:
```bash
for mod in apppublic ahserver sqlor appbase rbac; do
  cd ~/repos/$mod && git status --short
done
```
All reference modules should show clean status (no changes).