hotspot/SKILL.md

name	description	category
hotspot-radar	Build, extend, or troubleshoot the Hotspot Radar module for Sage — hotspot aggregation, parser config, analysis engine, and deployment.	devops

Hotspot Radar Module

When the user works with or asks about the Hotspot Radar platform (热点雷达), load this skill.

Architecture

• Repo: /Users/ymq/devops/hotspot → git@git.opencomputing.cn:yumoqing/hotspot.git
• Python package: hotspot/ with 4 modules:
  • parsers.py — 11 builtin platform parsers + 4 extractors (json_path/css/regex/rss)
  • engine.py — run_fetch() orchestrates source→fetch→extract→save→log
  • analysis.py — run_analysis() 5-dimension scoring + 7-day half-life classifier
  • db.py — save_items(), write_fetch_log(), get_stats()
• Sage integration: .dspy files are 1-line wrappers calling the Python package
• Tables (6): hotspot_source, hotspot_schedule, hotspot_fetch_log, hotspot_item, hotspot_analysis, hotspot_alert
• DB name: sage (matches Sage const.py)

Adding a New Builtin Parser

Edit hotspot/parsers.py, add at module bottom:

from hotspot.parsers import _reg
_reg('new_platform',
     'https://api.example.com/hot',
     lambda data: [
         {'title': item['name'], 'url': item['link'], 'heat_score': item['score']}
         for item in data.get('list', [])
     ],
     {'User-Agent': 'Mozilla/5.0'})

Then in Sage, add a source with:

{"parser": "builtin", "builtin_name": "new_platform"}

Adding a Custom Extractor Rule

In Sage → 来源管理 → 编辑 → parser_config:

{
  "parser": "json_path",
  "item_path": "$.result.items[*]",
  "field_map": {
    "title": "name",
    "url": "href",
    "heat_score": "popularity",
    "summary": "description",
    "category": "type"
  }
}

Deployment Checklist

1. cp -r hotspot $SAGE_ROOT/
2. cp json/hotspot_*.json $SAGE_ROOT/json/
3. cp -r wwwroot/hotspot $SAGE_ROOT/wwwroot/
4. Execute ddl/mysql.sql against Sage database
5. Append install/load_path_append.txt into load_path.py before closing """
6. Append install/menu_append.json into menu.ui items array
7. cd $SAGE_ROOT && python load_path.py
8. Restart Sage

Pitfalls

• Sage import path: The hotspot/ Python package must be at $SAGE_ROOT/hotspot/ so from hotspot.engine import run_fetch works in DSPY files.
• DB name: Always sage — never change the dbname in JSON table defs.
• parser_config JSON: Must be valid JSON. Malformed config → no items extracted → log shows items_total: 0.
• aiohttp timeout: All HTTP calls use 30s timeout. Slow sources will be logged as failed.
• CSS extractor limitation: Only extracts <a href="...">text</a> patterns. Full CSS needs BeautifulSoup4 (not included — Sage doesn't bundle it).

Verification

cd /Users/ymq/devops/hotspot
python3 -c "
import ast, json
for f in ['hotspot/__init__.py','hotspot/parsers.py','hotspot/engine.py','hotspot/analysis.py','hotspot/db.py']:
    ast.parse(open(f).read()); print(f'OK {f}')
for t in ['hotspot_source','hotspot_schedule','hotspot_fetch_log','hotspot_item','hotspot_analysis','hotspot_alert']:
    json.load(open(f'json/{t}.json')); print(f'OK {t}')
print('All OK')
"