yumoqing/pipeline-service
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity
30 Commits 1 Branch 0 Tags
Commit Graph

30 Commits

This Branch
This Branch
All Branches
Author SHA1 Message Date
yumoqing
48f5048df8 fix: shell_exec 目录不存在时回落可写基础目录 2026-08-05 18:01:17 +08:00
yumoqing
1f848e0760 fix: _resolve_workdir 先检查父目录存在+可写再 mkdir 2026-08-05 17:58:53 +08:00
yumoqing
222bdbcd3d fix: _resolve_workdir → /tmp 回落（apitest无/d写权限） 2026-08-05 17:58:00 +08:00
yumoqing
5a7104aa5d feat(v3.2.1): shell_exec + skill_import_git — 主agent devops能力 
- shell_exec: 安全子进程执行（沙箱在 /d/pipeline/workspaces 下，60s超时）
- skill_import_git: git clone + skills目录扫描，返回技能列表
- 注册到 env，cockpit_chat 可通过 dspy 直接调用
 2026-08-05 17:56:22 +08:00
yumoqing
ee9ab066fc fix: on_startup handler spawn 后台任务后立即返回（不阻塞 aiohttp 启动） 2026-08-05 17:31:27 +08:00
yumoqing
99cc19f9aa fix: _role_poller 接受 app 参数（aiohttp on_startup 信号） 2026-08-05 17:28:23 +08:00
yumoqing
0bd9609532 fix: add_startup 传函数引用（非协程对象） 2026-08-05 17:27:24 +08:00
yumoqing
4c0b349718 feat: 后台轮询器 — add_startup 自动执行 role_task 
- _role_poller 每10秒扫描 pipeline_tasks 中 submitted 的 role_task
- 逐条 dispatch 给对应角色 agent_loop 异步执行
- 修复 agent_loop 中 agent_ask 导入为相对导入
 2026-08-05 17:20:26 +08:00
yumoqing
4cb406003e feat(v3.2.0): 主agent+角色agent架构 — role任务认领/问题回路 
- pipeline_tasks 增加 role/claimed_by 字段+索引（存量库需ALTER）
- agent_loop 重写为角色循环：原子认领(role匹配+令牌校验)→LLM执行→交付/need_info提问
- 角色别名归一化(developer→develop等)，ROLE_PROMPT改replace防花括号崩溃
- questions.py 新增问题回路：agent_ask/answer_question/forward_question/list_questions
- init.py 注册 pipeline_role_submit 与问题回路 env 接口
 2026-08-05 12:58:52 +08:00
yumoqing
20896b6fa0 docs: add README.md and skill/SKILL.md 2026-08-03 16:46:19 +08:00
yumoqing
09e58e5e19 fix: DBPools init (config.databases), sor.R→sqlExe, add call_llm() wrapper 
- _get_db() / _get_task_raw(): properly init DBPools with config.databases
- Replace sor.R() with sqlExe() to avoid 3-arg signature issues
- Add call_llm() as SDLC handler interface delegating to llm_call
 2026-08-03 10:38:09 +08:00
yumoqing
9eaf6c2e74 fix: dict access → getattr, sor.R() 3-arg → sqlExe in storage.py 2026-08-02 16:37:48 +08:00
yumoqing
f172889fbb fix: sor.R() 3rd arg → sqlExe with ORDER BY 2026-08-02 15:51:59 +08:00
yumoqing
59e56b5e9c feat: Phase 2+4 — agent_loop engine + project agent config 2026-08-02 14:59:32 +08:00
yumoqing
0423fa3c99 refactor: add intent_classifier.py, register in init.py, LLM bridge shared 2026-08-02 00:00:20 +08:00
yumoqing
0253615fcc fix: decrypt api_key from llm table in llm_bridge 2026-08-01 16:32:47 +08:00
yumoqing
d535e61f29 fix: llm_bridge reads model config from llm DB table, falls back to env vars 2026-08-01 10:25:33 +08:00
yumoqing
23047eed95 feat: add pipeline_restart() - restart completed/failed/cancelled tasks with new version 2026-07-08 21:03:20 +08:00
yumoqing
99fd9753b2 fix: CRUD endpoint return format - remove json.dumps, use widgettype format 2026-07-03 14:50:43 +08:00
Hermes Agent
a44130b23b refactor(v3.1): decouple KTV handlers — pipeline-service is pure engine 
- Remove handlers_ktv.py (943 lines) — KTV is now an external adapter
- Remove register_ktv_handlers() from load_pipeline_service()
- Remove KTV imports from __init__.py
- pipeline-service no longer knows about any specific pipeline type
- Version bump: 3.0.0 → 3.1.0

Each pipeline type should be a separate adapter package that calls:
  register_handler(step_type, fn)
  register_step_type(step_type, metadata)

This keeps pipeline-service stable and lets adapters evolve independently.
 2026-06-16 11:11:05 +08:00
Hermes Agent
b9a5810d85 feat(v3): human-in-the-loop — interactive steps, pluggable step_type registry 
- New states: waiting (step/task), rejected (step)
- New tables: pipeline_human_tasks, pipeline_step_types
- New module: step_registry.py — pluggable step_type metadata
- New module: human.py — human_complete, approval_approve, approval_reject
- Executor: detects interactive step_types, creates human_tasks, enters waiting
- Reject with rollback: approval_reject(rollback_to=step) resets steps and re-runs
- API: human_task_complete, approval_approve, approval_reject, human_task_list
- API: pipeline_step_types, pipeline_register_step_type, pipeline_unregister_step_type
- Built-in interactive types: human_task, approval_gate
- Updated DDL and appcodes
 2026-06-16 11:05:45 +08:00
yumoqing
54c98acc25 fix: list_tasks用sqlExe替代sort参数 2026-06-12 00:20:01 +08:00
yumoqing
113eb7e040 feat: add KTV pipeline handlers (17 step types, 3 production modes) 
- handlers_ktv.py: 17 async step handlers for KTV production
  - Audio/Video preparation (ffmpeg)
  - Demucs vocal separation (GPU server SSH)
  - Lyric calibration (SenseVoice ASR + LLM)
  - Subtitle rendering (ASS karaoke format)
  - Lyric generation & evaluation (Mode C)
  - Music generation (Suno/MiniMax API)
  - Character design & image generation (wan2.7)
  - Storyboard generation (LLM)
  - Scene video generation (T2V/Ref2V)
  - Scene video evaluation (quality threshold)
  - Scene video concatenation (ffmpeg loop)
  - KTV synthesis (dual-track + MTV)

- llm_bridge.py: async LLM call bridge (harnessed_agent / OpenAI API)
- storage.py: extract deps from step_config JSON
- init.py: auto-register KTV handlers on load
 2026-06-11 20:36:05 +08:00
yumoqing
19cd63e719 fix: remove invalid default for deps text field 2026-06-11 18:50:53 +08:00
yumoqing
8e4a8c47f3 remove useless load_path.py (backend-only module, no paths to register) 2026-06-11 18:41:59 +08:00
yumoqing
0dea29e3af fix: 移除 sage 引用，改为通用 APP_ROOT 2026-06-11 17:46:45 +08:00
yumoqing
2448ad45f7 refactor: 改造为通用产线执行引擎模块 
- 去掉独立 aiohttp 服务器，改为标准模块（load_pipeline_service）
- 存储从文件系统改 MySQL（sqlor）
- 新增 3 张数据表：pipeline_tasks/task_steps/artifacts
- 多租户隔离（tenant_id）
- 通用 DAG 调度引擎（读 pipeline_steps 表，不硬编码业务）
- 可插拔步骤处理器（register_handler by step_type）
- artifact 版本管理 + 级联重跑
- init/data.json 标准 appcodes 格式
- 完整 README 文档
 2026-06-11 17:30:06 +08:00
yumoqing
5124b4a047 docs: 补全README — 引擎架构/状态/关联仓库 2026-06-11 14:59:11 +08:00
yumoqing
fd6d982704 feat: pipeline-service早期产线引擎(初始版本) 2026-06-11 14:47:08 +08:00
yumoqing
f48c97c852 Initial commit 2026-06-10 22:42:02 +08:00