76 lines
2.8 KiB
Markdown
76 lines
2.8 KiB
Markdown
---
|
||
name: hotspot-radar
|
||
description: Hotspot aggregation & analysis module for Sage — 11 builtin parsers, 5-dimension analysis
|
||
category: devops
|
||
---
|
||
|
||
# Hotspot Radar Module
|
||
|
||
指定来源的热点聚合分析平台。11 个内置平台解析器，4 种可配置提取器，五维分析 + 7天半衰期自动分类。
|
||
|
||
## Architecture
|
||
|
||
- **Repo**: `git@git.opencomputing.cn:yumoqing/hotspot.git`
|
||
- **Tables** (6): `hotspot_source`, `hotspot_schedule`, `hotspot_fetch_log`, `hotspot_item`, `hotspot_analysis`, `hotspot_alert`
|
||
- **Python package**: `hotspot/` — `init.py` registers on ServerEnv, `parsers.py` has 11 builtins
|
||
- **DSPY**: `import json` + `get_sor_context(env, 'sage')` + `env.run_fetch(sor)`
|
||
|
||
## Builtin Parsers (11)
|
||
|
||
| parser_name | Platform | API Endpoint |
|
||
|---|---|---|
|
||
| weibo_hot | 微博热搜 | weibo.com/ajax/side/hotSearch |
|
||
| zhihu_hot | 知乎热榜 | zhihu.com/api/v3/feed/topstory/hot-lists |
|
||
| baidu_hot | 百度热搜 | top.baidu.com/board |
|
||
| toutiao_hot | 头条热榜 | toutiao.com/hot-event/hot-board |
|
||
| bilibili_hot | B站热门 | api.bilibili.com/x/web-interface/popular |
|
||
| douyin_hot | 抖音热点 | 第三方聚合接口 |
|
||
| 36kr_hot | 36氪热榜 | 36kr.com/api/search/list/info-flow/hot |
|
||
| github_trending | GitHub | api.github.com/search/repositories |
|
||
| hackernews | HackerNews | firebaseio.com (两步) |
|
||
| v2ex_hot | V2EX | v2ex.com/api/topics/hot.json |
|
||
| rsshub | RSSHub | 来源 url 即地址 |
|
||
|
||
## Extractor Types (4)
|
||
|
||
| parser | 场景 | parser_config 示例 |
|
||
|--------|------|---------|
|
||
| `builtin` | 内置平台 | `{"parser":"builtin","builtin_name":"weibo_hot"}` |
|
||
| `json_path` | REST API | `{"parser":"json_path","item_path":"$.data.list[*]","field_map":{...}}` |
|
||
| `css` | HTML | `{"parser":"css","item_selector":".hot-item","field_map":{...}}` |
|
||
| `regex` | 文本 | `{"parser":"regex","item_regex":"(?P<title>...)"}` |
|
||
| `rss` | RSS/Atom | `{"parser":"rss"}` |
|
||
|
||
## 5-Dimension Analysis
|
||
|
||
| 维度 | 评分依据 |
|
||
|------|---------|
|
||
| time (时效) | hours_alive: <1h=95, <6h=85, <24h=70 |
|
||
| heat (热度) | heat_score: >10k=95, >5k=85, >1k=70 |
|
||
| content (内容) | title_len + summary_len + tags + category |
|
||
| propagation (传播) | engagement + comments×2 + shares×3 |
|
||
| audience (受众) | total_engagement / heat_score ratio |
|
||
|
||
## Status Classification
|
||
|
||
```
|
||
decay = e^(-hours_alive / 168) # 7天半衰期
|
||
adjusted_heat = raw_heat × decay
|
||
hours > 336 → expired
|
||
168 < hours ≤ 336 → cooling
|
||
adjusted_heat > 5000 → hot
|
||
adjusted_heat > 500 → rising (velocity>50) else emerging
|
||
other → emerging
|
||
```
|
||
|
||
## Deployment
|
||
|
||
```bash
|
||
cp -r hotspot $SAGE_ROOT/
|
||
cp models/hotspot_*.json $SAGE_ROOT/models/
|
||
cp -r wwwroot/hotspot $SAGE_ROOT/wwwroot/
|
||
mysql -u user -p sage < ddl/mysql.sql
|
||
python scripts/load_path.py
|
||
# sage.py: from hotspot.init import load_hotspot + load_hotspot()
|
||
```
|