diff --git a/README.md b/README.md index 4e4229a..50f90f3 100644 --- a/README.md +++ b/README.md @@ -1,2 +1,32 @@ # pipeline_core +产线核心模块 —— 管理产线定义、步骤编排、版本控制、发布流程。 + +## 功能 + +- **产线管理**:产线 CRUD,定义产线名称与描述 +- **步骤编排**:产线步骤的增删改查,支持依赖关系(DAG) +- **版本控制**:产线版本发布与回滚 +- **编辑器**:可视化产线步骤编排界面 + +## 数据表 + +| 表 | 说明 | +|---|------| +| pipelines | 产线定义 | +| pipeline_steps | 产线步骤(含 deps 依赖) | +| pipeline_versions | 产线版本快照 | +| llm | 大模型配置 | + +## 安装 + +```bash +cd pkgs/pipeline_core && pip install . +``` + +## 集成 + +伞仓 (pipeline-app) 通过 wwwroot symlink 加载本模块: +``` +wwwroot/pipeline_core -> ../pkgs/pipeline_core/wwwroot +``` diff --git a/skill/SKILL.md b/skill/SKILL.md new file mode 100644 index 0000000..169a696 --- /dev/null +++ b/skill/SKILL.md @@ -0,0 +1,63 @@ +--- +name: pipeline-core +description: "Pipeline core module — pipeline definition, step orchestration, version control, and publishing." +--- + +# pipeline_core + +Core module for managing pipeline definitions, step DAG orchestration, version snapshots, and publishing workflow. + +## Directory Layout + +``` +pipeline_core/ +├── pipeline_core/ # Python package +│ ├── __init__.py +│ └── init.py # ServerEnv registration via load_pipeline_core() +├── wwwroot/ +│ ├── index.ui # Module entry — pipeline list dashboard +│ ├── api/ +│ │ ├── pipelines_*.dspy # CRUD wrappers for pipelines table +│ │ ├── pipeline_steps_*.dspy # CRUD wrappers for pipeline_steps +│ │ ├── pipeline_versions_*.dspy +│ │ └── pipeline_publish.dspy # Version publish + activate +│ └── pipeline_editor/ +│ ├── index.ui # Visual step editor +│ └── save_steps.dspy # Save DAG configuration +├── models/ # Table definitions (JSON) +├── json/ # CRUD definitions +├── scripts/load_path.py # RBAC registration +└── pyproject.toml +``` + +## Data Model + +| Table | Key Fields | Purpose | +|-------|-----------|---------| +| `pipelines` | id, name, description | Pipeline definition | +| `pipeline_steps` | id, pipeline_id, step_name, step_type, step_order, deps (JSON) | Step DAG nodes | +| `pipeline_versions` | id, pipeline_id, version, config (JSON) | Version snapshots | +| `llm` | id, name, provider, model, api_key | LLM model config | + +## DSPY Endpoints + +### Pipeline Management +- `pipelines_create.dspy` / `pipelines_update.dspy` / `pipelines_delete.dspy` — standard CRUD +- `get_pipelines.dspy` — list with optional filters + +### Step Orchestration +- `pipeline_steps_create.dspy` / `pipeline_steps_update.dspy` / `pipeline_steps_delete.dspy` +- `get_pipeline_steps.dspy` — list steps by pipeline_id +- `get_step_types.dspy` — enumerate available step type handlers +- `save_steps.dspy` — batch save full DAG from editor UI + +### Version Control +- `pipeline_versions_create.dspy` / `pipeline_versions_update.dspy` / `pipeline_versions_delete.dspy` +- `pipeline_publish.dspy` — snapshot + activate version + +## Pitfalls + +- **sor.R() 3-arg → sqlExe**: Do NOT use `sor.R('table', where, sort)`. Use `sor.sqlExe("SELECT ... ORDER BY ...", params)`. +- **get_sor_context not DBPools().sqlorContext**: DSPY files must use `get_sor_context(env, 'pipeline')`. +- **CRUD generated dirs**: `wwwroot/pipeline_steps/`, `wwwroot/pipeline_versions/`, etc. are auto-generated by `xls2ui`. Never edit directly — modify `json/*.json` and re-run `build.sh`. +- **load_path.py**: Every new .dspy must be registered. No wildcards.