docs: add README.md and skill/SKILL.md
This commit is contained in:
parent
21ecaf3a5e
commit
66b8b42064
30
README.md
30
README.md
@ -1,2 +1,32 @@
|
||||
# pipeline_core
|
||||
|
||||
产线核心模块 —— 管理产线定义、步骤编排、版本控制、发布流程。
|
||||
|
||||
## 功能
|
||||
|
||||
- **产线管理**:产线 CRUD,定义产线名称与描述
|
||||
- **步骤编排**:产线步骤的增删改查,支持依赖关系(DAG)
|
||||
- **版本控制**:产线版本发布与回滚
|
||||
- **编辑器**:可视化产线步骤编排界面
|
||||
|
||||
## 数据表
|
||||
|
||||
| 表 | 说明 |
|
||||
|---|------|
|
||||
| pipelines | 产线定义 |
|
||||
| pipeline_steps | 产线步骤(含 deps 依赖) |
|
||||
| pipeline_versions | 产线版本快照 |
|
||||
| llm | 大模型配置 |
|
||||
|
||||
## 安装
|
||||
|
||||
```bash
|
||||
cd pkgs/pipeline_core && pip install .
|
||||
```
|
||||
|
||||
## 集成
|
||||
|
||||
伞仓 (pipeline-app) 通过 wwwroot symlink 加载本模块:
|
||||
```
|
||||
wwwroot/pipeline_core -> ../pkgs/pipeline_core/wwwroot
|
||||
```
|
||||
|
||||
63
skill/SKILL.md
Normal file
63
skill/SKILL.md
Normal file
@ -0,0 +1,63 @@
|
||||
---
|
||||
name: pipeline-core
|
||||
description: "Pipeline core module — pipeline definition, step orchestration, version control, and publishing."
|
||||
---
|
||||
|
||||
# pipeline_core
|
||||
|
||||
Core module for managing pipeline definitions, step DAG orchestration, version snapshots, and publishing workflow.
|
||||
|
||||
## Directory Layout
|
||||
|
||||
```
|
||||
pipeline_core/
|
||||
├── pipeline_core/ # Python package
|
||||
│ ├── __init__.py
|
||||
│ └── init.py # ServerEnv registration via load_pipeline_core()
|
||||
├── wwwroot/
|
||||
│ ├── index.ui # Module entry — pipeline list dashboard
|
||||
│ ├── api/
|
||||
│ │ ├── pipelines_*.dspy # CRUD wrappers for pipelines table
|
||||
│ │ ├── pipeline_steps_*.dspy # CRUD wrappers for pipeline_steps
|
||||
│ │ ├── pipeline_versions_*.dspy
|
||||
│ │ └── pipeline_publish.dspy # Version publish + activate
|
||||
│ └── pipeline_editor/
|
||||
│ ├── index.ui # Visual step editor
|
||||
│ └── save_steps.dspy # Save DAG configuration
|
||||
├── models/ # Table definitions (JSON)
|
||||
├── json/ # CRUD definitions
|
||||
├── scripts/load_path.py # RBAC registration
|
||||
└── pyproject.toml
|
||||
```
|
||||
|
||||
## Data Model
|
||||
|
||||
| Table | Key Fields | Purpose |
|
||||
|-------|-----------|---------|
|
||||
| `pipelines` | id, name, description | Pipeline definition |
|
||||
| `pipeline_steps` | id, pipeline_id, step_name, step_type, step_order, deps (JSON) | Step DAG nodes |
|
||||
| `pipeline_versions` | id, pipeline_id, version, config (JSON) | Version snapshots |
|
||||
| `llm` | id, name, provider, model, api_key | LLM model config |
|
||||
|
||||
## DSPY Endpoints
|
||||
|
||||
### Pipeline Management
|
||||
- `pipelines_create.dspy` / `pipelines_update.dspy` / `pipelines_delete.dspy` — standard CRUD
|
||||
- `get_pipelines.dspy` — list with optional filters
|
||||
|
||||
### Step Orchestration
|
||||
- `pipeline_steps_create.dspy` / `pipeline_steps_update.dspy` / `pipeline_steps_delete.dspy`
|
||||
- `get_pipeline_steps.dspy` — list steps by pipeline_id
|
||||
- `get_step_types.dspy` — enumerate available step type handlers
|
||||
- `save_steps.dspy` — batch save full DAG from editor UI
|
||||
|
||||
### Version Control
|
||||
- `pipeline_versions_create.dspy` / `pipeline_versions_update.dspy` / `pipeline_versions_delete.dspy`
|
||||
- `pipeline_publish.dspy` — snapshot + activate version
|
||||
|
||||
## Pitfalls
|
||||
|
||||
- **sor.R() 3-arg → sqlExe**: Do NOT use `sor.R('table', where, sort)`. Use `sor.sqlExe("SELECT ... ORDER BY ...", params)`.
|
||||
- **get_sor_context not DBPools().sqlorContext**: DSPY files must use `get_sor_context(env, 'pipeline')`.
|
||||
- **CRUD generated dirs**: `wwwroot/pipeline_steps/`, `wwwroot/pipeline_versions/`, etc. are auto-generated by `xls2ui`. Never edit directly — modify `json/*.json` and re-run `build.sh`.
|
||||
- **load_path.py**: Every new .dspy must be registered. No wildcards.
|
||||
Loading…
x
Reference in New Issue
Block a user