yumoqing
fa257f36b1
依据现有实现补全「自动识别故障 → 人工确认切换 → 切回」运维核心闭环。 触发模式 1A+2B+3A：VIP 浮动 + 半自动 + 切前隔离原主（防脑裂）。 【表结构】 clusters 加 vip/failover_mode；node_commands 加 command_type 新增 switchover_steps（切换步骤级审计） 【Agent 六动作（脚本化、幂等、结构化返回）】 fence(停MySQL+摘VIP)/unfence(恢复)/promote_master(STOP+RESET SLAVE) revert_promote(降回从库)/flip_sync(反转同步)/verify(可写+VIP校验) 【切换编排器 foms/failover.py】 六步状态机：precheck→fence→promote→flip_sync→verify→converge 失败 → 已完成步骤逆向回滚 → switchover_logs=failed 回滚也失败 → rollback_failed + 最高级告警冒泡人工 防脑裂不变量：fence 先于 promote，fence 失败硬中止绝不 promote 【健康判定器】 后台循环（默认5s）扫 active 集群，心跳超时/复制异常判定 防抖：连续 failover_threshold 次命中才判故障（防瞬时抖动误判） 半自动：只告警（写 switchover_logs + hostwatch_events）不切，等人确认 【切回 + 回滚语义】 trigger_switchback 反向六步；回滚用 unfence/revert_promote/反转/换回 【部署文档】 docs/deploy-operations.md：架构/三机部署/keepalived/Agent 环境变量/ 日常运维/失败语义/防脑裂/已知限制 验证：语法全过；编排器↔Agent 六命令类型对齐；六步顺序正确； 防脑裂不变量成立（fence idx=1 < promote idx=2）；注册链完整。
30 lines
1.5 KiB
JSON
30 lines
1.5 KiB
JSON
{
|
|
"summary": [
|
|
{
|
|
"name": "node_commands",
|
|
"title": "节点远程命令",
|
|
"primary": ["id"],
|
|
"catelog": "entity"
|
|
}
|
|
],
|
|
"fields": [
|
|
{"name": "id", "title": "命令ID", "type": "str", "length": 32, "nullable": "no"},
|
|
{"name": "node_id", "title": "节点ID", "type": "str", "length": 32, "nullable": "no"},
|
|
{"name": "command_type", "title": "命令类型", "type": "str", "length": 32, "nullable": "no", "default": "shell"},
|
|
{"name": "command", "title": "命令内容", "type": "text", "nullable": "no"},
|
|
{"name": "status", "title": "状态", "type": "str", "length": 16, "nullable": "no", "default": "pending"},
|
|
{"name": "exit_code", "title": "退出码", "type": "int"},
|
|
{"name": "output", "title": "执行输出", "type": "text"},
|
|
{"name": "created_at", "title": "下发时间", "type": "datetime", "nullable": "no"},
|
|
{"name": "executed_at", "title": "执行时间", "type": "datetime"}
|
|
],
|
|
"indexes": [
|
|
{"name": "idx_ncmd_node", "idxtype": "index", "idxfields": ["node_id"]},
|
|
{"name": "idx_ncmd_status", "idxtype": "index", "idxfields": ["status"]}
|
|
],
|
|
"codes": [
|
|
{"field": "status", "table": "appcodes_kv", "valuefield": "k", "textfield": "v", "cond": "parentid='cmd_status'"},
|
|
{"field": "command_type", "table": "appcodes_kv", "valuefield": "k", "textfield": "v", "cond": "parentid='cmd_type'"}
|
|
]
|
|
}
|