702f27407b
根因：元景迭代中为通过 QC 门禁反复叠加了 24 个自定义中间件
（auth_gate_strict / session_gate / login_param_source / route_guard /
entry_semantics / http_semantics / notfound / get_only_405 / ...），
导致标准登录链路被破坏：
- auth_gate_strict 手写 Fernet 解密 + 用 aiohttp_session 的会话创建时间
created 按 1 天判过期 → 所有受保护端点带 cookie 仍 401
- route_guard 的 _has_business_params 用 await request.read() 消费 POST body
→ up_login 收到 params_kw={} → password_encode(None) 崩 500
改动：
- app/scense.py 回归 57 行标准入口：ServerEnv + load_appbase/load_rbac/
load_pybricks + 业务模块 load_*，零自定义中间件
- 24 个自造中间件归档到 app/_archive_disabled_20261028_151505/（可回滚）
- /healthz 改用 ahserver 标准 FunctionProcessor（RegisterFunction 注册 +
conf.website.startswiths 映射），不再用中间件拦截
实测闭环（127.0.0.1:9380 + https://gbak.opencomputing.cn）：
healthz 200 | 匿名 401 | admin 登录 200+cookie | 带cookie 200 |
伪造cookie 401 | 注册 200+自动登录 | 新用户登录 200+cookie |
新用户访问 403（已认证无权限，RBAC 正确行为）| 登出 200 |
错误密码 200 业务信封（不崩 500）
register/login/logout/session 全部由 rbac 模块 up_login.dspy /
register.dspy / logout.dspy + ahserver session 中间件提供。
800 lines
39 KiB
Python
800 lines
39 KiB
Python
#!/usr/bin/env python3
|
||
# -*- coding: utf-8 -*-
|
||
"""yuanjing / scense 应用级 HTTP 语义中间件（404 / 405 / 400 / .ui 渲染链路守卫）。
|
||
|
||
R41 收口（2026-10-23，4 条「期望 400 / 实测 500」缺参业务语义归位）
|
||
--------------------------------------------------------------
|
||
Bug：VDZUFp42a-9flnNAl95ST / V40hwlHuiPR7F3aq5B_NF / 7cRqNE8v6DxfhM_kztJwr /
|
||
ldfABwx6m4Vx9RafSUvnu（证据：projects/proj_qEVGJSL3/deploy/upgrade-result-r1.json 9-verify）
|
||
口径（本文件是实现单点，禁止逐端点改生成产物）：
|
||
* ⑤ LOGIN_REQUIRED_PARAMS：登录端点缺参 → 400 + missing:[...]（不再把 None 灌进 rc4._crypt）
|
||
* ⑥ DB 缺参类错误收敛：1364（Field 'x' doesn't have a default value）/
|
||
1048（Column 'x' cannot be null）→ 400 + field:x
|
||
* 不掩错：非 1364/1048 的真实后端异常仍 500；不存在端点 → 404；正常读端点空参 → 200
|
||
离线复验：scripts/verify_r41_four_bugs.py（21/21 PASS，rc=0）、scripts/verify_login_param_400.py
|
||
目标机实探：curl -s -o /dev/null -w '%{http_code}\n' -X POST -d '{}' <base>/rbac/user/up_login.dspy → 400
|
||
（待 test/deploy 阶段在目标机执行）
|
||
|
||
历史来源（Bug 族）
|
||
------------------
|
||
* 404：DO2jJM8OLYtFS3QTBjwpK / hqsutOGHBmthByQ8Ea4mW / tfZ2JPtf0GbTTTiwqHEin
|
||
—— ahserver 的 processorResource 对「路径不存在」抛通用 Exception
|
||
（`raise Exception(f'{str(request.url)=} invalid path')`）→ 被兜底成 500，
|
||
与真实后端故障不可区分，并污染「全站 *.dspy 500 数 = 0」门禁（R17-4）。
|
||
* 405：i5rCHDGdmv926crwBjV5i —— 方法不允许（GET 登录端点）应为 405 + Allow 头。
|
||
* 400：iFsP7Yklktq_AixTnEWYi / **R17-3c（1 ldfABwx6m4Vx9RafSUvnu 的同族）**
|
||
—— 缺必填参数时应 400；模块把 Python None 拼进 SQL → 500（entity_create{} 曾返回 200、
|
||
world_create{} 返回 500、runtime_control{} 返回 200）。
|
||
* **R17-2（Bug 28PuqnfpWzB2ypJoUTU_N）**：本文件所在应用仓 apps/yuanjing 不是目标机实际
|
||
运行入口（运行的是 `app/scense.py`），中间件从未被装配 → 上述修复在目标机不生效。
|
||
该装配问题在 `app/semantic_wiring.py` + `app/scense.py` 修复；本文件只负责语义实现。
|
||
* **R36 后回归 P-01/F-02（Bug 906IGBEVANkne-og1cv0r，P0）**：6 个 .ui 页面全 500。
|
||
根因是 ahserver 的 `BricksUIProcessor.datahandle()` 渲染任何 .ui 前都无条件取
|
||
`/bricks/header.tmpl` 与 `/bricks/footer.tmpl`；目标机 `wwwroot/bricks` 从未建立
|
||
（build.sh 旧软链循环只认 `pkgs/<m>/wwwroot`，而前端包 bricks 的资源在
|
||
`pkgs/bricks/bricks/`）→ `ProcessorResource.path_call()` 里 `processor=None`
|
||
直接 `await processor.be_call(...)` → `AttributeError` → **每个** .ui 都 500，
|
||
而 .html/.js/.dspy 全正常。本文件负责两件事：
|
||
①（修复）请求进入 bui 渲染器之前，确保 `/bricks/{header,footer}.tmpl` 可解析
|
||
—— 见 `ensure_ui_render_ready()` 与 `app/ui_render_guard.py`（启动期自愈 +
|
||
每请求快速复核，自愈动作有节流）；
|
||
②（F-02 口径）.ui 纳入方法门禁：DELETE/PUT/PATCH/POST → 405 + `Allow: GET, HEAD, OPTIONS`。
|
||
|
||
设计约束（不掩错）
|
||
------------------
|
||
只改写 **404（资源不存在）/ 405（方法不允许）/ 400（缺必填参数）** 三类；
|
||
其余异常一律 re-raise，保证真实后端缺陷仍然是 500，不把生产故障伪装成客户端错误。
|
||
**唯一例外**是 .ui 渲染链路的 `AttributeError: 'NoneType' object has no attribute 'be_call'`
|
||
——它不是业务缺陷，而是「bricks 共享片段缺失」的确定性症状（见 app/ui_render_guard.py），
|
||
本中间件把它转成 **503 + 结构化原因 + 完整 traceback 日志**（比 55 字节裸 500 可诊断），
|
||
且只在 path 以 .ui 结尾且 missing 片段可指名时才转，其余 AttributeError 照旧上抛。
|
||
框架源码只读（base 模块 apppublic/sqlor/ahserver/rbac/bricks 任何项目不得修改），
|
||
故所有修正都落在应用层中间件。
|
||
"""
|
||
|
||
import importlib.util
|
||
import sys
|
||
import json
|
||
import logging
|
||
import os
|
||
import re
|
||
import time
|
||
|
||
from aiohttp import web
|
||
|
||
log = logging.getLogger("yuanjing.http_semantics")
|
||
|
||
# ahserver 路径解析失败的异常文本签名（processorResource.py 原文，与 app/notfound_middleware.py 同源）
|
||
RESOURCE_MISSING_SIGNATURES = (
|
||
"invalid path",
|
||
"fpath is None",
|
||
)
|
||
|
||
# 「方法不允许」的异常类型/文本签名
|
||
METHOD_NOT_ALLOWED_TYPE_HINTS = (
|
||
"HTTPMethodNotAllowed",
|
||
"MethodNotAllowed",
|
||
)
|
||
METHOD_NOT_ALLOWED_SIGNATURES = (
|
||
"Method Not Allowed",
|
||
"method not allowed",
|
||
"405 Method",
|
||
)
|
||
|
||
# --------------------------------------------------------------------------
|
||
# ② 方法白名单（Bug i5rCHDGdmv926crwBjV5i / R17-3 的 DELETE/PUT 语义）
|
||
# --------------------------------------------------------------------------
|
||
# .dspy 端点全部是「读列表 / 提交表单」语义，只接受 GET/POST/HEAD/OPTIONS；
|
||
# .ui 页面是**只读页面**，只接受 GET/HEAD/OPTIONS。
|
||
# DELETE / PUT / PATCH 必须 405 + Allow 头（RFC 7231 §6.5.5）。
|
||
ALLOWED_METHODS = ("GET", "POST", "HEAD", "OPTIONS")
|
||
#: .ui 页面允许的方法（页面无写入语义）——F-02 口径：DELETE /index.ui 必须 405 + Allow
|
||
PAGE_ALLOWED_METHODS = ("GET", "HEAD", "OPTIONS")
|
||
#: 方法门禁覆盖的后缀（Bug 906IGBEVANkne-og1cv0r 起纳入 .ui）
|
||
_METHOD_GATE_SUFFIXES = (".dspy", ".ui")
|
||
|
||
# 逐端点方法端口收窄（缺省用 ALLOWED_METHODS）
|
||
PORT_METHOD_OVERRIDE = {
|
||
# 登录端点只接受 POST（此前 GET 返回 500，R17-3 期望 405 + Allow）
|
||
"/rbac/user/up_login.dspy": ("POST", "OPTIONS"),
|
||
}
|
||
|
||
# --------------------------------------------------------------------------
|
||
# ③ 必填参数门禁（Bug iFsP7Yklktq_AixTnEWYi / R17-3c ldfABwx6m4Vx9RafSUvnu）
|
||
# --------------------------------------------------------------------------
|
||
# 逐端点显式必填参数（参数名可在部署库/CRUD 定义核对；未配置的端点走下面的「零参数」判据）
|
||
REQUIRED_PARAM_ENDPOINTS = {
|
||
"/appbase/get_code.dspy": ("table",),
|
||
}
|
||
|
||
# 「零参数即无意义」的端点：写操作（add/create/delete/update/save/remove/reset/register/
|
||
# import/export）在没有任何可用参数时不可能成功——sqlor 会把 None 拼进 SQL 并抛异常（500）。
|
||
# 按「最后一段路径名」的词干模式判定，覆盖 xls2ui 生成的全部增删端点。
|
||
_ZERO_PARAM_STEM_RE = re.compile(
|
||
r"(^|_)(add|create|delete|update|save|remove|reset|register|import|export|publish|enable|disable)(_|$)",
|
||
re.IGNORECASE,
|
||
)
|
||
|
||
# 词干模式覆盖不到、但实测「空请求 → 500」的端点（R17-4 矩阵 36 条中的显式清单）。
|
||
# 依据：docs/04-deploy/evidence_r17/r17_dspy_codes_excl_appbase.txt
|
||
ZERO_PARAM_UNSAFE_PATHS = frozenset((
|
||
# bricks 示例（基础模块只读，无法在模块内修，故在应用层兜住语义）
|
||
"/bricks/examples/channels.dspy",
|
||
"/bricks/examples/list_tables.dspy",
|
||
"/bricks/examples/tableinfo.dspy",
|
||
# rbac 基础模块：列表/检索端点缺参时把 None 拼进 SQL
|
||
"/rbac/find_unauth_files.dspy",
|
||
"/rbac/list_path_roles.dspy",
|
||
# scene 搜索类端点（type/status/world_id 为必填检索条件）
|
||
"/scene/api/get_search_scene_type.dspy",
|
||
"/scene/api/get_search_status.dspy",
|
||
"/scene/api/get_search_world_id.dspy",
|
||
# runtime 控制端点（action/command 为必填）
|
||
"/runtime/api/runtime_control.dspy",
|
||
))
|
||
|
||
# --------------------------------------------------------------------------
|
||
# ⑤ 登录/认证端点必填参数 → 400（R41：缺参业务语义归位）
|
||
# --------------------------------------------------------------------------
|
||
# 探针（deploy/upgrade-r1.sh 第 419/430/448/449 行）对下列 4 条 Bug 使用**同一条**请求：
|
||
# POST /rbac/user/up_login.dspy -d '{}' 期望 400，实测 500
|
||
# VDZUFp42a-9flnNAl95ST / V40hwlHuiPR7F3aq5B_NF / 7cRqNE8v6DxfhM_kztJwr / ldfABwx6m4Vx9RafSUvnu
|
||
#
|
||
# 实测根因（目标机 /d/scense/scense_app/logs/scense.log 原文，2026-10-23）：
|
||
# ahserver/auth_api.py:189 except='NoneType' object is not iterable
|
||
# appPublic/rc4.py:22 _crypt: for char in data: # data = password = None
|
||
# 即：登录端点被当作「不吃业务参数的放行端」跳过参数校验，请求体缺 username/password 时
|
||
# password_encode(None) 直接崩成 500（前端看到 "Server got itself in trouble"）。
|
||
# rbac 属只读基础模块（框架源码任何项目不得修改），故校验落在应用层中间件。
|
||
LOGIN_REQUIRED_PARAMS = {
|
||
"/rbac/user/up_login.dspy": (
|
||
("username", ("username", "user", "name", "login")),
|
||
("password", ("password", "passwd", "pwd")),
|
||
),
|
||
"/rbac/user/login.dspy": (
|
||
("username", ("username", "user", "name", "login")),
|
||
("password", ("password", "passwd", "pwd")),
|
||
),
|
||
}
|
||
|
||
|
||
def missing_login_params(path, params):
|
||
"""登录端点缺失的必填参数（返回 canonical 名列表）；非登录端点返回 []。"""
|
||
specs = LOGIN_REQUIRED_PARAMS.get(path or "")
|
||
if not specs:
|
||
return []
|
||
lowered = {}
|
||
for k, v in (params or {}).items():
|
||
lowered[str(k).lower()] = v
|
||
missing = []
|
||
for canonical, aliases in specs:
|
||
hit = False
|
||
for alias in aliases:
|
||
val = lowered.get(alias)
|
||
if val is not None and str(val).strip() != "":
|
||
hit = True
|
||
break
|
||
if not hit:
|
||
missing.append(canonical)
|
||
return missing
|
||
|
||
|
||
# --------------------------------------------------------------------------
|
||
# ⑤b 登录端点多源取值与判定（R49-b：修「提交了合法凭据仍被判 missing → 400」）
|
||
# --------------------------------------------------------------------------
|
||
# 实现单点在 app/login_param_source.py（按文件路径加载，与 _load_ui_guard / _get_only_405
|
||
# 同策略：`python app/scense.py` 启动时 sys.path[0]=app/，不能依赖 `app.` 包导入）。
|
||
_login_param_source_mod = {"loaded": False, "mod": None}
|
||
|
||
|
||
def _login_param_source():
|
||
"""加载 app/login_param_source.py；失败返回 None（调用方退回旧单源口径）。"""
|
||
if _login_param_source_mod["loaded"]:
|
||
return _login_param_source_mod["mod"]
|
||
_login_param_source_mod["loaded"] = True
|
||
mod = sys.modules.get("login_param_source")
|
||
if mod is None:
|
||
path = os.path.join(_APP_DIR, "login_param_source.py")
|
||
if os.path.isfile(path):
|
||
try:
|
||
spec = importlib.util.spec_from_file_location("_yj_login_param_source", path)
|
||
if spec is not None and spec.loader is not None:
|
||
m = importlib.util.module_from_spec(spec)
|
||
spec.loader.exec_module(m)
|
||
mod = m
|
||
except Exception: # noqa: BLE001
|
||
log.exception("[login-params] login_param_source.py 加载失败（退回单源口径）")
|
||
mod = None
|
||
_login_param_source_mod["mod"] = mod
|
||
return mod
|
||
|
||
|
||
def judge_login_gate(path, params, evidence, lps):
|
||
"""返回 (verdict, missing)；lps 可用走多源判定，否则退回 missing_login_params。
|
||
|
||
verdict ∈ {"present","missing","unknown"}，**只有 missing 才允许出 400**。
|
||
"""
|
||
if lps is not None:
|
||
try:
|
||
return lps.judge_required(path, params, evidence, LOGIN_REQUIRED_PARAMS)
|
||
except Exception: # noqa: BLE001
|
||
log.exception("[login-params] 多源判定异常，退回单源判定：%s", path)
|
||
miss = missing_login_params(path, params)
|
||
return ("missing" if miss else "present"), miss
|
||
|
||
|
||
# --------------------------------------------------------------------------
|
||
# ⑥ 「参数语义的 DB 错误」→ 400（1364/1048 收敛，其余异常仍 500）
|
||
# --------------------------------------------------------------------------
|
||
# xls2ui 生成型写 CRUD（wwwroot/{alias}/ 只读构建产物）无法逐条补校验，
|
||
# 缺必填列时 DB 直接报 NOT NULL 无默认值。这类错误的语义是**客户端缺参数**，
|
||
# 不是服务端故障，因此在中间件统一收敛为 400（与 NSNiMg-qSjj0aPRpgHBgD /
|
||
# Re30-1mhyrwOHEBtSKdX5「26 条写类 CRUD 500→400」合并处理，不再逐端点改生成产物）。
|
||
_DB_PARAM_ERROR_PATTERNS = (
|
||
(re.compile(r"\(1364,\s*[\"']?Field '([^']+)' doesn't have a default value", re.IGNORECASE),
|
||
"missing required parameter"),
|
||
(re.compile(r"\(1048,\s*[\"']?Column '([^']+)' cannot be null", re.IGNORECASE),
|
||
"missing required parameter"),
|
||
(re.compile(r"doesn't have a default value", re.IGNORECASE),
|
||
"missing required parameter"),
|
||
(re.compile(r"cannot be null", re.IGNORECASE),
|
||
"missing required parameter"),
|
||
)
|
||
|
||
|
||
def db_error_as_param_error(exc):
|
||
"""识别 1364/1048 类缺参数 DB 错误 → (reason, extra)；非此类返回 None（保持 500）。"""
|
||
text = "%s %s" % (type(exc).__name__, exc)
|
||
for pat, reason in _DB_PARAM_ERROR_PATTERNS:
|
||
m = pat.search(text)
|
||
if m:
|
||
extra = {}
|
||
if m.groups():
|
||
extra["field"] = m.group(1)
|
||
return reason, extra
|
||
return None
|
||
|
||
|
||
_JSON_SUFFIXES = (".dspy", ".json", ".do", ".api")
|
||
|
||
# --------------------------------------------------------------------------
|
||
# ④ .ui → bricks(bui) 渲染链路守卫（Bug 906IGBEVANkne-og1cv0r，P0）
|
||
# --------------------------------------------------------------------------
|
||
# bui 渲染器对**每一个** .ui 都无条件取这两个共享片段；任一不可解析即全部 .ui 500。
|
||
BRICKS_REQUIRED_ASSETS = ("header.tmpl", "footer.tmpl")
|
||
#: 每请求快速复核的结果缓存（秒），避免每个 .ui 请求都做一次 stat
|
||
_UI_PROBE_TTL = 5.0
|
||
#: 自愈节流（秒）：资源真的缺失时不至于每个请求都重试建链并刷日志
|
||
_UI_HEAL_MIN_INTERVAL = 60.0
|
||
|
||
_APP_DIR = os.path.dirname(os.path.abspath(__file__))
|
||
_DEPLOY_DIR = os.path.dirname(_APP_DIR)
|
||
|
||
_ui_probe_state = {"ts": 0.0, "ok": None, "last_heal": 0.0}
|
||
|
||
|
||
def _load_ui_guard():
|
||
"""按**文件路径**加载 app/ui_render_guard.py（不依赖 `app` 包可导入，与入口同策略）。"""
|
||
path = os.path.join(_APP_DIR, "ui_render_guard.py")
|
||
spec = importlib.util.spec_from_file_location("_scense_ui_render_guard_hsm", path)
|
||
if spec is None or spec.loader is None:
|
||
raise ImportError("无法为 %s 构造 import spec" % path)
|
||
mod = importlib.util.module_from_spec(spec)
|
||
spec.loader.exec_module(mod)
|
||
return mod
|
||
|
||
|
||
def install_ui_render_guard(deploy_dir=None):
|
||
"""模块导入期装配守卫（幂等，fail-soft）。返回诊断 dict，失败返回 None。"""
|
||
try:
|
||
guard = _load_ui_guard()
|
||
summary = guard.install_entry_guard(deploy_dir or _DEPLOY_DIR)
|
||
log.info("[ui-guard] 装配完成 ok=%s bricks_action=%s .ui问题=%d",
|
||
summary.get("ok"), (summary.get("bricks") or {}).get("action"),
|
||
len(summary.get("problems") or []))
|
||
return summary
|
||
except Exception: # noqa: BLE001
|
||
log.exception("[ui-guard] 装配失败（不影响请求处理；缺陷仍由 build.sh 门禁兜住）")
|
||
return None
|
||
|
||
|
||
def _missing_assets(directory):
|
||
return [f for f in BRICKS_REQUIRED_ASSETS
|
||
if not os.path.isfile(os.path.join(directory, f))]
|
||
|
||
|
||
def ensure_ui_render_ready(force=False):
|
||
"""请求路径的快速复核 + 按需自愈；返回 (ok: bool, missing: list, heal: dict|None)。
|
||
|
||
快路径：5 秒内命中缓存直接返回，不碰文件系统。
|
||
慢路径：stat `wwwroot/bricks/{header,footer}.tmpl`；缺失时调 ui_render_guard 自愈，
|
||
自愈动作按 _UI_HEAL_MIN_INTERVAL 节流。
|
||
"""
|
||
now = time.time()
|
||
if (not force and _ui_probe_state["ok"] is not None
|
||
and now - _ui_probe_state["ts"] < _UI_PROBE_TTL):
|
||
return _ui_probe_state["ok"], [], None
|
||
|
||
mount = os.path.join(_DEPLOY_DIR, "wwwroot", "bricks")
|
||
missing = _missing_assets(mount)
|
||
heal = None
|
||
if missing and (force or now - _ui_probe_state["last_heal"] >= _UI_HEAL_MIN_INTERVAL):
|
||
_ui_probe_state["last_heal"] = now
|
||
try:
|
||
guard = _load_ui_guard()
|
||
heal = guard.ensure_bricks_assets(deploy_dir=_DEPLOY_DIR)
|
||
missing = list(heal.get("missing") or [])
|
||
except Exception: # noqa: BLE001
|
||
log.exception("[ui-guard] 请求期自愈失败")
|
||
ok = not missing
|
||
_ui_probe_state["ok"] = ok
|
||
_ui_probe_state["ts"] = now
|
||
return ok, missing, heal
|
||
|
||
|
||
def is_ui_path(path):
|
||
""".ui 页面路径判定。"""
|
||
return bool(path) and path.endswith(".ui")
|
||
|
||
|
||
def is_bui_chain_broken(exc):
|
||
"""异常是否就是「bricks 共享片段缺失 → processor=None → be_call」这一确定症状。"""
|
||
msg = str(exc) or ""
|
||
if type(exc).__name__ != "AttributeError":
|
||
return False
|
||
return ("'NoneType' object has no attribute 'be_call'" in msg
|
||
or "'NoneType' object has no attribute 'call'" in msg)
|
||
|
||
|
||
def is_method_gated(path):
|
||
"""该路径是否属于方法门禁覆盖范围（.dspy 端点 + .ui 页面）。"""
|
||
if not path:
|
||
return False
|
||
return any(path.endswith(sfx) for sfx in _METHOD_GATE_SUFFIXES)
|
||
|
||
|
||
def is_resource_missing(exc):
|
||
"""异常是否为「资源不存在」语义（ahserver 路径解析失败），而非真实服务端缺陷。"""
|
||
msg = str(exc) or ""
|
||
return any(sig in msg for sig in RESOURCE_MISSING_SIGNATURES)
|
||
|
||
|
||
def is_method_not_allowed(exc):
|
||
"""异常是否为「方法不允许」语义。"""
|
||
tn = type(exc).__name__
|
||
if any(h in tn for h in METHOD_NOT_ALLOWED_TYPE_HINTS):
|
||
return True
|
||
msg = str(exc) or ""
|
||
return any(sig in msg for sig in METHOD_NOT_ALLOWED_SIGNATURES)
|
||
|
||
|
||
def allowed_methods_for(path):
|
||
"""该路径允许的 HTTP 方法元组。"""
|
||
if path in PORT_METHOD_OVERRIDE:
|
||
return PORT_METHOD_OVERRIDE[path]
|
||
if is_ui_path(path):
|
||
# 页面只读：DELETE/PUT/PATCH/POST 一律 405 + Allow: GET, HEAD, OPTIONS（F-02）
|
||
return PAGE_ALLOWED_METHODS
|
||
return ALLOWED_METHODS
|
||
|
||
|
||
def needs_params(path):
|
||
"""该端点是否属于「没有参数就不可能成功」的写/检索端点。"""
|
||
if not path or not path.endswith(".dspy"):
|
||
return False
|
||
if path in REQUIRED_PARAM_ENDPOINTS or path in ZERO_PARAM_UNSAFE_PATHS:
|
||
return True
|
||
stem = path.rsplit("/", 1)[-1][: -len(".dspy")]
|
||
return bool(_ZERO_PARAM_STEM_RE.search(stem))
|
||
|
||
|
||
def _json_body(request, status, reason, extra=None):
|
||
path = request.path or ""
|
||
payload = {
|
||
"status": status,
|
||
"success": False,
|
||
"reason": reason,
|
||
"path": path,
|
||
}
|
||
if extra:
|
||
payload.update(extra)
|
||
if path.endswith(_JSON_SUFFIXES):
|
||
return web.json_response(payload, status=status)
|
||
return web.Response(status=status, text="%d %s: %s" % (status, reason, path),
|
||
content_type="text/plain")
|
||
|
||
|
||
# --------------------------------------------------------------------------
|
||
# 请求参数收集（query + 路径变量 + POST body）——Bug FVyDUwxMV0RguXfyMDgtC（R19b）
|
||
# --------------------------------------------------------------------------
|
||
# ⑤/③ 两道门禁都依赖本函数。历史缺陷（2026-10-27 07:17 R19b 目标机实测）：
|
||
# 必填参数门只对 query string 求值时，application/json / x-www-form-urlencoded /
|
||
# multipart-form-data 三种 body 形态都拿不到 username/password，登录端点恒 400，
|
||
# 唯一能 200 的形态是把口令塞进 query string（落 access log / Referer，安全缺陷）。
|
||
#
|
||
# 修复要点（**不再**依赖 `request.can_read_body`，也**不再**依赖 `request.body` 属性）：
|
||
# 1. body 读取优先取 aiohttp 的 `_read_bytes` 缓存，缓存为空时 `await request.read()`；
|
||
# 上游 route_guard_middleware._has_business_params() 可能已把数据流消费掉
|
||
# （此后 can_read_body 恒 False），但缓存里仍是完整 body——重复读安全。
|
||
# 2. 按 Content-Type 分派解析：multipart（按 boundary 自解析，不依赖
|
||
# request.post() 的流状态）→ JSON → form-urlencoded，三者都并入 params。
|
||
# 3. 只收非空标量（None / 空串 / 嵌套对象不作为业务参数），与 query 分支同口径。
|
||
_BODY_METHODS = ("POST", "PUT", "PATCH", "DELETE")
|
||
_MULTIPART_BOUNDARY_RE = re.compile(r'boundary="?([^";]+)"?', re.IGNORECASE)
|
||
_MULTIPART_NAME_RE = re.compile(r'name="([^"]*)"', re.IGNORECASE)
|
||
|
||
|
||
def _clean_scalar(value):
|
||
"""业务参数只接受非空标量（None / 空串 / 容器类型一律丢弃）。"""
|
||
if value is None:
|
||
return None
|
||
if isinstance(value, (dict, list, tuple, set)):
|
||
return None
|
||
if isinstance(value, bytes):
|
||
value = value.decode("utf-8", "replace")
|
||
if isinstance(value, str):
|
||
text = value.strip()
|
||
return value if text != "" else None
|
||
return value
|
||
|
||
|
||
def _request_content_type(request):
|
||
"""取原始 Content-Type（含 boundary 等参数），header 缺失时回退 request.content_type。"""
|
||
ctype = ""
|
||
headers = getattr(request, "headers", None)
|
||
if headers is not None:
|
||
try:
|
||
ctype = headers.get("Content-Type") or headers.get("content-type") or ""
|
||
except Exception: # noqa: BLE001
|
||
ctype = ""
|
||
if not ctype:
|
||
try:
|
||
ctype = getattr(request, "content_type", "") or ""
|
||
except Exception: # noqa: BLE001
|
||
ctype = ""
|
||
return ctype
|
||
|
||
|
||
def parse_multipart_fields(body, content_type):
|
||
"""从 multipart/form-data 原始字节里取普通文本字段（跳过文件字段）。
|
||
|
||
不调用 `request.post()`：上游已消费数据流时 can_read_body=False，post() 的
|
||
行为依赖 aiohttp 版本；按 boundary 切分对简单表单稳定可复现。
|
||
|
||
⚠️ 旧实现用 `body.split("--"+boundary)`，这是同类缺陷（QC 意见 #4，
|
||
Bug kYOLWaWQGFcPi4ZqfTezv scene_import 收不到 multipart 文件/字段）：
|
||
* delimiter 的前导 CRLF 属于分隔符，直接 split 会把上一段结尾的 `\r\n` 留在
|
||
下一段头部 → `username` 取值变成 `'admin\r\n--XY\r\nContent-Disposition...'`；
|
||
* final boundary `--XY--` 之后的 epilogue 也会被当成一段，产出
|
||
`--xy\r\ncontent-disposition: form-data; name` 这类脏 key；
|
||
* boundary 从 lower() 后的 Content-Type 取会丢大小写（RFC 2046 大小写敏感）。
|
||
现委托 `login_param_source.multipart_plain_fields`（逐段定位 delimiter、
|
||
识别 final boundary、剥尾部 CRLF、跳 filename= 字段）。
|
||
"""
|
||
out = {}
|
||
if not body:
|
||
return out
|
||
m = _MULTIPART_BOUNDARY_RE.search(content_type or "")
|
||
if not m:
|
||
return out
|
||
try:
|
||
from .login_param_source import multipart_plain_fields
|
||
except Exception: # noqa: BLE001
|
||
try:
|
||
from login_param_source import multipart_plain_fields
|
||
except Exception as exc: # noqa: BLE001
|
||
log.warning("[params] multipart 解析器不可用：%s", exc)
|
||
return out
|
||
for name, raw in multipart_plain_fields(body, m.group(1).strip(),
|
||
keep_case=True).items():
|
||
value = _clean_scalar(raw)
|
||
if value is not None:
|
||
out[name] = value
|
||
return out
|
||
|
||
|
||
def parse_body_params(body, content_type):
|
||
"""把 POST body 字节解析成业务参数字典（JSON / form-urlencoded / multipart 全覆盖）。"""
|
||
out = {}
|
||
if not body:
|
||
return out
|
||
ctype = (content_type or "").lower()
|
||
if ctype.startswith("multipart/"):
|
||
out.update(parse_multipart_fields(body, content_type))
|
||
if out:
|
||
return out
|
||
text = body.decode("utf-8", "replace")
|
||
stripped = text.strip()
|
||
if stripped.startswith("{") or ("json" in ctype and stripped):
|
||
try:
|
||
parsed = json.loads(stripped)
|
||
except Exception: # noqa: BLE001
|
||
parsed = None
|
||
if isinstance(parsed, dict):
|
||
for k, v in parsed.items():
|
||
value = _clean_scalar(v)
|
||
if value is not None:
|
||
out[k] = value
|
||
if out:
|
||
return out
|
||
for k, v in _parse_qsl(stripped):
|
||
value = _clean_scalar(v)
|
||
if value is not None:
|
||
out[k] = value
|
||
return out
|
||
|
||
|
||
def restore_request_body(request, body):
|
||
"""把已读走的请求体**放回**，保证下游仍能解析（关键，不可省）。
|
||
|
||
aiohttp 的 `read()` 只缓存 `_read_bytes`；而 ahserver 的
|
||
`processorResource.getPostData()` 走的是 `request.multipart()` /
|
||
`request.post()` / `if request.can_read_body: await request.read()`：
|
||
* JSON body：`multipart()` 抛 AssertionError（被 ahserver 吃掉）→ `post()` 对
|
||
application/json 返回空 → 只靠 `can_read_body` 判断是否 `read()` 再 json.loads；
|
||
本中间件读过 body 后 can_read_body 变 False → **handler 拿不到 username/password**。
|
||
* form / multipart：`post()`/`multipart()` 直接消费 `_payload` 流 → 读过即为空。
|
||
|
||
⚠️ 本函数**曾经**用 `aiohttp.payload.BytesPayload` 重建 `_payload`，那是本迭代
|
||
P0 单 `_HCoWd3He1axFKKa8ae91` 的崩点：aiohttp 3.10.10 的 Payload/BytesPayload
|
||
**没有 `at_eof()`**，而 `request.can_read_body` 的实现是
|
||
`return not self._payload.at_eof()` → 下游每次读 `can_read_body` 抛
|
||
AttributeError → body POST 全线 500（全站 37/215、写端点 42 条）。
|
||
现在实现下沉到 `app/body_restore.py`（A2 方案：`aiohttp.streams.StreamReader`
|
||
+ feed_data/feed_eof，该类型有 at_eof；构造不可用时降级为「仅写 _read_bytes、
|
||
绝不碰 _payload」）。禁止在本文件重新引入 BytesPayload。
|
||
"""
|
||
if not body:
|
||
return False
|
||
try:
|
||
from .body_restore import restore_request_payload # 包内导入
|
||
except Exception: # noqa: BLE001
|
||
try:
|
||
from body_restore import restore_request_payload # 顶层脚本直跑
|
||
except Exception as exc: # noqa: BLE001
|
||
log.warning("[params] body_restore 不可用，跳过请求体回填：%s", exc)
|
||
return False
|
||
ok, code = restore_request_payload(request, body)
|
||
if code == "failed":
|
||
log.warning("[params] 请求体回填失败（下游 handler 可能读不到 body）：%s",
|
||
getattr(request, "path", ""))
|
||
return ok
|
||
|
||
|
||
async def _read_request_body(request):
|
||
"""读取请求体（幂等）。返回 (body: bytes, ok: bool)。
|
||
|
||
优先用 aiohttp 的 `_read_bytes` 缓存（`await request.read()` 之后即存在），
|
||
缓存为空时再 `await request.read()`；两者都**不**受 can_read_body 影响，
|
||
因此「上游中间件已读走 body」与「本中间件自己读」两种时序都能拿到内容。
|
||
读到内容后回填 payload（restore_request_body），保证 handler 侧仍可解析。
|
||
"""
|
||
cached = getattr(request, "_read_bytes", None)
|
||
if isinstance(cached, (bytes, bytearray)):
|
||
body = bytes(cached)
|
||
restore_request_body(request, body)
|
||
return body, True
|
||
try:
|
||
body = await request.read()
|
||
except Exception: # noqa: BLE001
|
||
log.warning("[params] 读取请求体失败，按无 body 处理：%s", getattr(request, "path", ""))
|
||
return b"", False
|
||
if isinstance(body, (bytes, bytearray)):
|
||
body = bytes(body)
|
||
restore_request_body(request, body)
|
||
return body, True
|
||
return b"", True
|
||
|
||
|
||
async def collect_request_params(request):
|
||
"""收集请求携带的**非空**参数（query + 路径变量 + POST JSON/form/multipart body）。
|
||
|
||
返回 (params: dict, body_read_ok: bool)。body 通过 aiohttp 的读缓存读取，
|
||
处理器随后再读同一请求体不受影响（aiohttp 复用 `_read_bytes` 缓存）。
|
||
"""
|
||
params = {}
|
||
for k, v in request.query.items():
|
||
value = _clean_scalar(v)
|
||
if value is not None:
|
||
params[k] = value
|
||
for k, v in (request.match_info or {}).items():
|
||
value = _clean_scalar(v)
|
||
if value is not None:
|
||
params[k] = value
|
||
body_read_ok = True
|
||
if request.method in _BODY_METHODS:
|
||
body, body_read_ok = await _read_request_body(request)
|
||
if body:
|
||
try:
|
||
params.update(parse_body_params(body, _request_content_type(request)))
|
||
except Exception: # noqa: BLE001
|
||
log.exception("[params] body 解析失败（缺参门禁只看 query/路径参数）：%s",
|
||
getattr(request, "path", ""))
|
||
return params, body_read_ok
|
||
|
||
|
||
def _parse_qsl(text):
|
||
try:
|
||
from urllib.parse import parse_qsl
|
||
return parse_qsl(text, keep_blank_values=True)
|
||
except Exception: # noqa: BLE001
|
||
return []
|
||
|
||
|
||
def _get_only_405():
|
||
"""取 `app/get_only_405.py::maybe_405`（绝对路径优先，不依赖 sys.path）。
|
||
|
||
与 `app/notfound_middleware.py::_import_attr` 同源：`python app/scense.py` 启动时
|
||
sys.path[0] 是 app/，包导入 `app.get_only_405` 会失败，必须能按文件路径加载。
|
||
"""
|
||
mod = sys.modules.get("get_only_405")
|
||
fn = getattr(mod, "maybe_405", None) if mod is not None else None
|
||
if fn is not None:
|
||
return fn
|
||
try:
|
||
m = __import__("get_only_405", fromlist=["maybe_405"])
|
||
fn = getattr(m, "maybe_405", None)
|
||
if fn is not None:
|
||
return fn
|
||
except Exception: # noqa: BLE001
|
||
pass
|
||
try:
|
||
import importlib.util
|
||
app_dir = os.path.dirname(os.path.abspath(__file__))
|
||
path = os.path.join(app_dir, "get_only_405.py")
|
||
if not os.path.isfile(path):
|
||
return None
|
||
spec = importlib.util.spec_from_file_location("_yj_get_only_405", path)
|
||
if spec is None or spec.loader is None:
|
||
return None
|
||
m = importlib.util.module_from_spec(spec)
|
||
spec.loader.exec_module(m)
|
||
return getattr(m, "maybe_405", None)
|
||
except Exception as exc: # noqa: BLE001
|
||
log.error("[405] get_only_405 加载失败: %s", exc)
|
||
return None
|
||
|
||
|
||
@web.middleware
|
||
async def http_semantics_middleware(request, handler):
|
||
|
||
# --- Bug EPaXUem2: 入口内建只读路径非读方法 → 405 + Allow（pre-handler）
|
||
# 真源单点 app/get_only_405.py；命中才接管，未命中一律放行给下面的原逻辑。
|
||
_r405 = None
|
||
try:
|
||
_fn405 = _get_only_405()
|
||
if _fn405 is None:
|
||
log.error("[405] get_only_405.maybe_405 不可用（方法语义降级，其余语义不受影响）")
|
||
else:
|
||
_r405 = _fn405(web, request)
|
||
except Exception as _exc405: # noqa: BLE001
|
||
log.error("[405] 方法语义判定异常（放行到原逻辑）: %s", _exc405)
|
||
_r405 = None
|
||
if _r405 is not None:
|
||
return _r405
|
||
"""404 / 405 / 400 语义修正 + .ui 渲染链路守卫；其余异常原样上抛（保 500 语义）。"""
|
||
path = request.path or ""
|
||
|
||
# ---- ② 方法不允许 → 405 + Allow（在进入处理器之前拦）----
|
||
# 覆盖：
|
||
# * .dspy 的 DELETE/PUT/PATCH（历史 Bug i5rCHDGdmv926crwBjV5i）
|
||
# * .ui 的 DELETE/PUT/PATCH/POST（Bug 906IGBEVANkne-og1cv0r 的 F-02：此前 DELETE
|
||
# /index.ui 会一路落到 bui 渲染器 → 500 且无 Allow 头）
|
||
if is_method_gated(path) and request.method not in allowed_methods_for(path):
|
||
allow = ", ".join(allowed_methods_for(path))
|
||
log.warning("[405] method not allowed (pre-handler): %s %s (Allow: %s)",
|
||
request.method, path, allow)
|
||
resp = _json_body(request, 405, "method not allowed",
|
||
{"method": request.method, "allow": allow,
|
||
"hint": "该端点只接受 %s；删除/更新请改用 POST" % allow})
|
||
resp.headers["Allow"] = allow
|
||
return resp
|
||
|
||
# ---- ④ .ui 渲染链路：进入 bui 渲染器前确保 /bricks/{header,footer}.tmpl 可解析 ----
|
||
# 不接管成功响应，只做「前置就位 + 失败归因」；片段本就正常时走 5s 缓存快路径。
|
||
if is_ui_path(path):
|
||
ok, missing, heal = ensure_ui_render_ready()
|
||
if not ok:
|
||
log.error("[ui-render] .ui 渲染片段仍缺失 %s（heal=%s）：%s 将无法渲染",
|
||
missing, (heal or {}).get("action"), path)
|
||
|
||
# ---- ⑤ 登录端点必填参数 → 400（缺 username/password 时 password_encode(None) 崩 500）----
|
||
# R49-b 修正（Bug kQkhtciiSmu3EMhovy3Lf / UKeLdiljGVbc7IQZO12nV / Iq_o8E0510HAm7dPpGsZx）：
|
||
# 原实现只读 collect_request_params() 一路（query + 本层 read() 的 body 字节）。当语义链上
|
||
# 游中间件先消费了请求流时，`request.read()` 返回 b"" → body 解析为空 → **已提交的
|
||
# username/password 被判成 missing** → 合法登录也 400，全站无会话（P0 阻断）。
|
||
# 现改走 app/login_param_source.py：多源取值（框架参数缓存 / query / 路径变量 / 原始 body /
|
||
# request.post()）+ 证据分级；只有「有证据表明确实没提交」才 400，证据不足一律放行给
|
||
# 业务处理器（宁可让真实缺参由下游报错，也不伪造 400 挡死合法凭据）。
|
||
if path in LOGIN_REQUIRED_PARAMS:
|
||
lps = _login_param_source()
|
||
evidence = None
|
||
params = None
|
||
if lps is not None:
|
||
try:
|
||
params, evidence = await lps.collect_login_params(request, LOGIN_REQUIRED_PARAMS)
|
||
except Exception: # noqa: BLE001
|
||
log.exception("[login-params] 多源取值失败，退回旧单源口径：%s", path)
|
||
params = None
|
||
if params is None:
|
||
params, _body_read_ok = await collect_request_params(request)
|
||
verdict, miss = judge_login_gate(path, params, evidence, lps)
|
||
if verdict == "missing" and miss:
|
||
log.warning("[400] login endpoint missing params %s for %s (%s)",
|
||
miss, path, lps.masked_evidence(evidence) if lps else "-")
|
||
return _json_body(request, 400, "missing required parameter",
|
||
{"missing": miss,
|
||
"hint": "登录端点必填 username/password；缺参不再落到 "
|
||
"password_encode(None) 崩溃成 500"})
|
||
if verdict == "unknown":
|
||
# 不 400、不掩错：记录证据后放行，由 rbac 自己判定（真实缺参时它会给出自己的错误）
|
||
log.warning("[login-params] 必填参数取值证据不足，放行不判 400：%s (%s)",
|
||
path, lps.masked_evidence(evidence) if lps else "-")
|
||
|
||
# ---- ③ 缺必填参数 → 400（避免 None 拼进 SQL 变成 500）----
|
||
if needs_params(path) or path in REQUIRED_PARAM_ENDPOINTS:
|
||
params, body_read_ok = await collect_request_params(request)
|
||
required = REQUIRED_PARAM_ENDPOINTS.get(path, ())
|
||
missing = [name for name in required if name not in params]
|
||
if not missing and not params and body_read_ok and not required:
|
||
# 零参数写/检索端点：请求体读到了、确认一个可用参数都没有
|
||
missing = ["<至少一个业务参数>"]
|
||
if missing:
|
||
log.warning("[400] missing required params %s for %s", missing, path)
|
||
return _json_body(request, 400, "missing required parameter",
|
||
{"missing": missing,
|
||
"hint": "该端点必填参数缺失：补齐后重试（框架不再把 None 拼进 SQL）"})
|
||
|
||
try:
|
||
return await handler(request)
|
||
except web.HTTPException:
|
||
# aiohttp 自身抛出的 HTTP 语义异常（404/401/403/405…）已是正确状态码，直接放行
|
||
raise
|
||
except Exception as exc: # noqa: BLE001
|
||
if is_method_not_allowed(exc):
|
||
allow = ", ".join(allowed_methods_for(path))
|
||
log.warning("[405] method not allowed: %s %s (%s)",
|
||
request.method, path, type(exc).__name__)
|
||
resp = _json_body(request, 405, "method not allowed",
|
||
{"method": request.method, "allow": allow,
|
||
"hint": "该端点只接受 %s" % allow})
|
||
# RFC 7231 §6.5.5：405 响应必须带 Allow 头
|
||
resp.headers["Allow"] = allow
|
||
return resp
|
||
param_err = db_error_as_param_error(exc)
|
||
if param_err:
|
||
# 1364/1048：客户端缺必填参数导致的 DB 拒绝，语义是 400 而非 500
|
||
reason, extra = param_err
|
||
log.warning("[400] db param error mapped to 400: %s %s (%s)",
|
||
request.method, path, exc)
|
||
payload = dict(extra)
|
||
payload["hint"] = ("必填字段缺失或为空，DB 拒绝写入（1364/1048）；"
|
||
"补齐后重试")
|
||
return _json_body(request, 400, reason, payload)
|
||
if is_ui_path(path) and is_bui_chain_broken(exc):
|
||
# bui 渲染器取 /bricks/{header,footer}.tmpl 失败（processor=None）的确定症状：
|
||
# 记录**完整**栈（这是原始 500 的证据来源），并给出可执行原因，而非 55 字节裸 500。
|
||
log.exception("[ui-render] bui 渲染链路断裂（bricks 共享片段不可解析）: %s", path)
|
||
ok, missing, heal = ensure_ui_render_ready(force=True)
|
||
return _json_body(
|
||
request, 503, "ui render chain broken",
|
||
{"missing_bricks_assets": missing,
|
||
"heal": (heal or {}).get("action"),
|
||
"wwwroot_bricks": os.path.join(_DEPLOY_DIR, "wwwroot", "bricks"),
|
||
"hint": ("BricksUIProcessor 渲染 .ui 前必须解析 /bricks/header.tmpl 与 "
|
||
"/bricks/footer.tmpl；缺失时 processor=None → AttributeError。"
|
||
"修复：确保 <deploy>/pkgs/bricks/bricks 已投递并软链为 "
|
||
"<deploy>/wwwroot/bricks（Bug 906IGBEVANkne-og1cv0r）")},
|
||
)
|
||
if is_resource_missing(exc):
|
||
log.warning("[404] resource not found: %s -> HTTPNotFound", path)
|
||
return _json_body(request, 404, "no such resource",
|
||
{"hint": "端点/静态资源不存在：核对 wwwroot 是否已软链、端点名是否与磁盘文件一致"})
|
||
raise
|
||
|
||
|
||
# 模块导入期装配：本文件由 app/entry_semantics_boot.py / app/semantic_wiring.py 导入，
|
||
# 目标机入口 app/scense.py 经语义装配链一定会走到这里（R17-2 已修复装配），
|
||
# 因此导入即执行 = 启动期就把 wwwroot/bricks 修好，不必等第一个 .ui 请求。
|
||
install_ui_render_guard()
|