---
name: build-script-modularization
title: Build Script Modularization and Database Separation
description: Guidelines for separating database setup from application build scripts and implementing modular processing loops
---

# Build Script Modularization and Database Separation

## Problem Statement
Build scripts often fail in production environments because they attempt to create databases using root privileges, which is both a security risk and often fails due to permission restrictions. Additionally, repetitive code for processing multiple modules reduces maintainability.

## Solution Approach

### 1. Separate Database Setup
- Create an external `setup_database.sh` script that handles:
  - Database creation
  - User creation  
  - Permission grants
  - This script requires root privileges and should be run separately

### 2. External Password Encryption
- Create a dedicated encryption script (`encrypt_password.py`) that:
  - Uses the application's encryption library (e.g., apppublic)
  - Reads existing config files to get encryption keys
  - Updates configuration with encrypted passwords
  - Provides fallback mechanisms for encryption failures

### 3. Modular Processing Loop
- Define modules as an array: `MODULES=("module1" "module2" "module3")`
- Use a for loop to process each module uniformly:
  - Clone/install modules
  - Generate database DDL from models
  - Generate CRUD UI from JSON definitions
  - Handle module-specific logic with conditional checks

### 4. Three-Step Deployment Process
1. **Database Setup**: Run external database script with root privileges
2. **Configuration Encryption**: Encrypt sensitive data and update configs
3. **Application Build**: Run main build script without elevated privileges

## Implementation Template

### setup_database.sh
```bash
#!/usr/bin/env bash
set -e
DB_NAME="your_db"
DB_USER="your_user" 
DB_PASS="secure_password"

mysql -u root -e "CREATE DATABASE IF NOT EXISTS ${DB_NAME} CHARACTER SET utf8mb4 COLLATE utf8mb4_unicode_ci;"
mysql -u root -e "CREATE USER IF NOT EXISTS '${DB_USER}'@'localhost' IDENTIFIED BY '${DB_PASS}';"
mysql -u root -e "GRANT ALL PRIVILEGES ON ${DB_NAME}.* TO '${DB_USER}'@'localhost';"
mysql -u root -e "FLUSH PRIVILEGES;"
```

### encrypt_password.py
```python
#!/usr/bin/env python3
import json
import sys

def encrypt_password(password, key):
    # Use application-specific encryption library
    # Provide fallback if library unavailable
    pass

# Read config, encrypt password, update config file
```

### build.sh (modular version)
```bash
#!/usr/bin/env bash
set -e

MODULES=("appbase" "rbac" "contract_management")

for modulename in "${MODULES[@]}"; do
    echo "Processing ${modulename}..."
    # Uniform processing logic for all modules
done
```

## Benefits
- **Security**: No root privileges needed during application build
- **Maintainability**: Single code path for all modules
- **Reliability**: Clear separation of concerns reduces failure points
- **Reusability**: Pattern applies to any multi-module application

## Common Pitfalls
- Forgetting to update config files after password encryption
- Not handling module-specific installation differences (git vs local copy)
- Assuming MySQL connection details are always available during build
- Missing error handling for missing model or JSON files
- **Case sensitivity issues**: On Linux systems, Python import statements are case-sensitive. `from appPublic.jsonconfig` will fail if the actual file is `jsonConfig.py`. Always verify exact module names by inspecting the source repository structure.

## Sage Module Build Pattern

For Sage platform modules, follow this standard build.sh structure:

### Standard Sage build.sh Template
```bash
#!/bin/bash
set -e

SCRIPT_DIR="$(cd "$(dirname "${BASH_SOURCE[0]}")" && pwd)"

# Find Sage root (try multiple candidates)
SAGE_ROOT=""
for candidate in "$SCRIPT_DIR/../.." "$HOME/repos/sage" "$HOME/sage"; do
    if [ -d "$candidate/wwwroot" ] && [ -d "$candidate/py3/bin" ]; then
        SAGE_ROOT="$(cd "$candidate" && pwd)"
        break
    fi
done

if [ -z "$SAGE_ROOT" ]; then
    echo "ERROR: Sage root not found"
    exit 1
fi

echo "Sage root: $SAGE_ROOT"

# Generate DDL from model JSON files
if [ -d "$SCRIPT_DIR/models" ]; then
    echo "Generating DDL from models..."
    cd "$SCRIPT_DIR/models"
    if ls *.json 1>/dev/null 2>&1; then
        "$SAGE_ROOT/py3/bin/json2ddl" mysql . > mysql.ddl.sql
        echo "DDL generated: models/mysql.ddl.sql"
    fi
fi

# Link wwwroot to Sage via symlink (not copy)
echo "Linking wwwroot to Sage..."
rm -f "$SAGE_ROOT/wwwroot/<module_name>"
ln -sf "$SCRIPT_DIR/wwwroot" "$SAGE_ROOT/wwwroot/<module_name>"

echo "<Module Name> build complete."
```

### Key Sage Build Patterns
- **Sage Root Detection**: Check for both `wwwroot/` and `py3/bin/` directories to confirm valid Sage installation
- **DDL Generation**: Use `$SAGE_ROOT/py3/bin/json2ddl mysql .` in the models/ directory to generate SQL from JSON model definitions
- **wwwroot Linking**: Always use symlinks (`ln -sf`), never copy files - this keeps modules in sync during development
- **Model Directory**: JSON model files live in `models/*.json`, DDL output goes to `models/mysql.ddl.sql`
- **Module Naming**: Replace `<module_name>` with your actual module name (e.g., `sage_datamart`, `dashboard_for_sage`)

### Examples from Sage Codebase
- `supplychain/build.sh` - includes CRUD UI generation from json/ definitions
- `dashboard_for_sage/build.sh` - includes pip install and API file linking