---
name: gpu-server-services
description: "Complete map of GPU server services on opencomputing.net — ports, nginx routing, systemd units, and operational commands."
version: 1.0.0
tags: [gpu-server, infrastructure, media, opencomputing, services, nginx, systemd]
trigger_conditions:
  - Working on opencomputing.net GPU server
  - Need to find which service runs on which port
  - Troubleshooting service failures or restarting the media pipeline
  - Planning new services that need to coexist with existing ones
---

# GPU Server Services — opencomputing.net

## Server Overview

- **Host**: opencomputing.net (ymq@, passwordless SSH + sudo)
- **GPU**: 8×RTX4090 24GB
- **Entry**: nginx :10443 with SNI domain routing
- **Infra**: Redis :6379, MySQL :3306
- **Disk**: 664GB usable under /data/ymq/

## Service Map (by port)

### Currently Running (July 2026)

| Port | Service | Dir | Python Env | Model Path | Git Repo | Notes |
|------|---------|-----|------------|------------|----------|-------|
| 8886 | VDB向量库 | /data/ymq/vdb | wan22-service/py3 | /data/ymq/vdb/db/milvus.db | yumoqing/vdb | Milvus via ahserver. upsert/search/delete. **Sole VDB — 8887 removed** |
| 9080 | KTV媒体服务 | /data/ymq/media-server | vllm-0.8.5 | /data/ymq/models/MahmoudAshraf/mms-300m-1130-forced-aligner (字幕对齐) + LLM(API,字幕校准) | yumoqing/media-server | Central media hub, systemd. Routes: /subtitle/ /calibrate/ /merge-video/ /ktv/ |
| 9081 | 歌曲评分 | /data/ymq/songrate-service | wan22-service/py3 | 评分模型(内置) | yumoqing/songrate-service | KTV quality scoring |
| 9082 | Real-ESRGAN超分 | /data/ymq/realesrgan-service | wan22-service/py3 | /data/ymq/models/RealESRGAN_x2plus.pth | yumoqing/realesrgan-service | 3 workers |
| 9083 | Demucs音源分离 | /data/ymq/demucs-service | wan22-service/py3 | /data/ymq/.cache/torch/hub/checkpoints/955717e8-8726e21a.th | yumoqing/demucs-service | 4 workers |
| 9084 | KTV合成 | /data/ymq/ktv-synth-service | wan22-service/py3 | 无ML模型(纯ffmpeg) | yumoqing/ktv-synth-service | ffmpeg视频拼接+音频混流 |
| 9085 | RVC声音转换 | /data/ymq/rvc-service | venv | /data/ymq/rvc-models/ | RVC-Project/RVC-WebUI | Voice cloning |
| 9086 | CLIP向量化 | /data/ymq/clip_embedding | vllm-0.8.5 | /data/ymq/models/laion/CLIP-ViT-H-14-laion2B-s32B-b79K | yumoqing/clip_embedding | /api/embed |
| 9087 | 声纹Embedding | /share/ymq/run/voiceprint | 内置venv | /share/models/ecapa-tdnn | yumoqing/voiceprint | ECAPA-TDNN, GPU1, extract/verify submit+status. **Replaced T2T on this port** |
| 9090 | Reranker重排 | /data/ymq/bge-reranker | vllm-0.8.5 | /data/ymq/models/BAAI/bge-reranker-v2-m3 | yumoqing/bge-reranker | /api/rerank |
| 9091 | 人脸服务 | /data/ymq/face-service | wan22-service/py3 | /data/ymq/.insightface/models/buffalo_l | yumoqing/face-service | InsightFace buffalo_l |
| 9092 | 图数据库 | /data/ymq/graph-service | wan22-service/py3 | Neo4j内嵌(内存图) | yumoqing/graph-service | graph CRUD |
| 9093 | NER实体识别 | /data/ymq/ner-service | venv | /data/ymq/models/gliner-multitask-large-v0.5 | yumoqing/ner-service | GLiNER multilingual |
| 9908 | 视频评估 | /data/ymq/video-eval | vllm-0.8.5 | PSNR/SSIM/VMAF | yumoqing/video-eval | systemd |
| 9925 | FastWhisper ASR | /data/ymq/asr-service | aligner/py3 | /data/ymq/models/deepdml/faster-whisper-large-v3-turbo-ct2 | yumoqing/asr-service | GPU 6 |
| 9926 | VibeVoice ASR | /share/ymq/run/vibevoice-asr | vllm-0.8.5 | /share/models/VibeVoice-ASR-7B | yumoqing/vibevoice-asr | 7B BF16, GPU 0, 人声分离 |
| 9997 | Reranker模型层 | /share/run/reranker | vllm-0.8.5 | /share/models/BAAI/bge-reranker-v2-m3 | — | Qwen3-Reranker-0.6B raw |
| 11434 | Ollama | systemd | system | Ollama模型目录 | — | LLM server |

**Shared Python environments**: `/data/ymq/wan22-service` (yumoqing/wan22-service) and `/data/ymq/aligner` (yumoqing/aligner).

### Currently Down

| Port | Service | Notes |
|------|---------|-------|
| 9991 | 三元组抽取 | nginx config exists, no process |
| 9994 | FastVLM | nginx config exists, no process |
| 9995 | TTS语音合成 | nginx config exists, no process |
| 9087 | T2T文本生成 | **nginx config removed** — port taken by voiceprint |
| 8887 | VDB旧实例 | **Removed** — systemctl disabled, process killed, nginx cleaned — 8886 sole VDB |
| 9089-9106 | vLLM qwen3 | 18 instances all down |

## Systemd Units

All under `/etc/systemd/system/`:

```
embedding.service  — Qwen3-Embedding-0.6B, WorkDir /share/run/embeddings
reranker.service   — Qwen3-Reranker-0.6B, WorkDir /share/run/reranker
entities.service   — WorkDir /share/run/entities
triples.service    — WorkDir /share/run/triples (Type=forking)
milvus.service     — WorkDir /share/run/milvus (Type=forking, TimeoutStartSec=300)
neo4j.service      — Failed (exit code 1)
vdb.service        — Failed, disabled
clip.service       — Inactive (killed TERM)
rag.service        — Inactive (dead)
qwen3.service      — vLLM (from /share/run/qwen3)
qwen3coder.service
gemma4.service
fvlm.service
fastwhisper.service — Auto-restart loop, fails with exit code 2
nvidia-asr.service
ollama.service      — Active
media-server.service — Active (only service confirmed running)
aligner.service
comfyui.service
m2m.service
subtitler.service
songrate.service
f5tts.service
video-eval.service
connection.service
```

## Nginx Routing (all on :10443)

Nginx uses SNI-based virtual hosts. Each `<name>.opencomputing.net` maps to a backend.

### Standalone Domain Services

| Domain | Backend | Notes |
|--------|---------|-------|
| `vectordb.opencomputing.net` | localhost:8886 | Sole VDB — /milvus/ route removed, all traffic → 8886 |
| `embedding.opencomputing.net` | localhost:9086 | CLIP ViT-H-14 |
| `reranker.opencomputing.net` | localhost:9090 | BGE reranker |
| `graphdb.opencomputing.net` | localhost:9092 | Neo4j graph |
| `entities.opencomputing.net` | localhost:9093 | GLiNER NER |
| `ktv.opencomputing.net` | localhost:9080 + sub-paths | KTV media hub |
| `ollama.opencomputing.net` | localhost:11434 | Ollama LLM |
| `evaluate.opencomputing.net` | localhost:9908 | Video eval |

### media.opencomputing.net — Consolidated Media Services

ALL media/KTV services are routed under sub-paths of `media.opencomputing.net`:

| Sub-path | Port | Service |
|----------|------|---------|
| `/face/` | 9091 | 人脸服务 |
| `/asr/` | 9925 | FastWhisper ASR |
| `/vibevoice/` | 9926 | VibeVoice人声ASR |
| `/voiceprint/` | 9087 | 声纹Embedding |
| `/demucs/` | 9083 | Demucs音源分离 |
| `/realesrgan/` | 9082 | Real-ESRGAN超分 |
| `/synth/` | 9084 | KTV合成 |
| `/rvc/` | 9085 | RVC声音转换 |
| `/songrate/` | 9081 | 歌曲评分 |
| `/video-eval/` | 9908 | 视频评估 |

### Removed

- `t2t.opencomputing.net` — nginx config deleted, port 9087 taken by voiceprint

## RAG Pipeline Architecture

Source: `~/rag-pipeline/`. NOT running as a service — code only.

### Pipeline flow (defined in pipeline.py):
```
ingest: chunk → embed(CLIP :9086) → store(VDB :8886) → extract entities(LLM) → store(Graph :9092)
search: embed query → hybrid retrieve(vector+graph RRF) → rerank(BGE :9090) → generate(LLM)
```

### Plugin registry (~/rag-pipeline/plugins/registry.py):
- **embedding**: CLIP ViT-H-14 (dim=1024), BGE-M3 (not deployed)
- **vdb**: Milvus Lite (8886), Qdrant (not deployed)
- **graph**: NetworkX (9092), FalkorDB (blocked)
- **reranker**: BGE Reranker v2-m3 (9090)
- **face**: InsightFace buffalo_l (dim=512, 9091)
- **chunker**: recursive, sentence
- **retriever**: hybrid (vector+graph+RRF), vector_only

### API endpoints (port 9093 in code, but ner-service occupies this port):
- `/api/status`, `/api/ingest`, `/api/search`, `/api/pipelines`, `/api/plugins`

## Quick Diagnostics

Full inventory spreadsheet: `~/GPU_Services.xlsx` (24 services, 20 running, 4 stopped — generated from `ssh ymq@opencomputing.net`). Columns: 端口, Base URL, 服务名称, 系统服务名, 运行路径, Python环境, 远端仓库, 模型路径(本地), API说明, 服务功能说明, 状态. All model paths are local absolute filesystem paths.

```bash
# Which services are running right now
ssh ymq@opencomputing.net "sudo systemctl list-units --type=service --state=running"

# All listening ports and their processes
ssh ymq@opencomputing.net "ss -tlnp | grep LISTEN"

# GPU memory usage
ssh ymq@opencomputing.net "nvidia-smi --query-gpu=index,memory.used,memory.total --format=csv,noheader"

# Check a specific service
ssh ymq@opencomputing.net "curl -s http://localhost:9091/api/status"
```

## Pitfalls

### External access requires :10443 — port 443 times out
nginx exposes ONLY :10443 for external traffic (SNI-routed vhosts). Plain HTTPS (port 443) to any `*.opencomputing.net` service hangs until timeout (curl HTTP 000). Every client — DSPY ingestion code, scripts, tests — must use `https://<svc>.opencomputing.net:10443/...`. Audit with `grep -n 'opencomputing.net' <files> | grep -v ':10443'` (must return nothing). A bare `except: pass` around such calls hides the timeout completely — the classic symptom is downstream data silently missing (e.g. RAG chunks with empty vector_id). Some admin endpoints (e.g. VDB `/v1/listcollections`) may additionally 403 from non-whitelisted client IPs; call from rag.opencomputing.cn or the GPU server itself when that happens.

### Port 9093 conflict
ner-service (GLiNER FastAPI) occupies port 9093, but rag-pipeline's code also assumes port 9093. If rag-pipeline is deployed, pick a different port or consolidate.

### Model path mismatch for ner-service
GLiNER model expected at `/mnt/disk0/yumoqing/models/gliner-multitask-large-v0.5` — this path does not exist. Model needs to be downloaded or symlinked to the actual model location under `/data/ymq/models/`.

### Voiceprint (9087) POST returns 405 / multipart uploads fail

**Symptoms**: `curl -X POST http://localhost:9087/extract/submit -F 'file=@audio.wav'` returns `405 Method Not Allowed` with `Allow: GET,HEAD`, OR returns `{"error": "audio_file required", "kw": []}` (params_kw empty).

**Root causes** (two separate issues):

1. **`_allowed_methods` not updated**: aiohttp `StaticResource.__init__` sets `_allowed_methods = set(self._routes)`. `ProcessorResource.__init__` updates `_routes` but not `_allowed_methods`. The dispatcher sees `Allow: GET,HEAD`. Fix: add `self._allowed_methods = set(self._routes.keys())` after the last `_routes.update()`.

2. **`get_session_userinfo` crashes**: When auth middleware isn't installed, `auth.get_auth(request)` raises `RuntimeError('auth_middleware not installed')`. This crashes `getPostData` during multipart processing, so file uploads aren't saved to `params_kw`. Fix: wrap in try/except in `auth_api.py`.

See `references/ecapa-tdnn-voiceprint.md` for complete fix + recovery steps.

### Voiceprint startup: `nohup ... &` hangs SSH

On the GPU server, `nohup cmd &` or `setsid cmd &` hangs the SSH connection. Use `ssh -f ymq@opencomputing.net "cd /share/ymq/run/voiceprint && PYTHONPATH=... python3 ah.py -p 9087 >> logs/voiceprint.log 2>&1"` instead.
fastwhisper.service is in `activating (auto-restart)` with exit code 2. Its ExecStart points to `/d/ymq/run/fastwhisper/py3/bin/python ah.py` — but the home directory is `/data/ymq`, not `/d/ymq`. This path mismatch is likely the cause.

### vLLM instances all down
Ports 9089-9106 are all unresponsive. The t2t nginx upstream still lists them. Start with the skill `vllm-multi-instance-gpu` for the correct startup procedure — critical to also kill orphan `VLLM::EngineCore` processes first.

### Many services use /share/run/ not ~/
embedding, reranker, entities, triples, milvus, qwen3, qwen3coder all have their working directories under `/share/run/` with start.sh/stop.sh scripts. The ~/ directories contain the same service types but as ahserver-based implementations — these may be newer replacements for the /share/run/ versions.