yumoqing/ktv-synth-service
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity

Compare commits

merge into: yumoqing:master
Branches Tags
yumoqing:main
yumoqing:master
...
pull from: yumoqing:main
Branches Tags
yumoqing:master
yumoqing:main

No commits in common. "master" and "main" have entirely different histories.
master ... main

14 changed files with 1 additions and 833 deletions
Show Stats Expand all files Collapse all files

54
.gitignore vendored
View File

@ -1,54 +0,0 @@
# Python
__pycache__/
*.py[cod]
*$py.class
*.so
.Python
build/
develop-eggs/
dist/
downloads/
eggs/
.eggs/
lib/
lib64/
parts/
sdist/
var/
wheels/
*.egg-info/
.installed.cfg
*.egg
# Virtual environments
venv/
env/
ENV/
.venv
# Logs
*.log
nohup.out
# Temporary files
*.tmp
*.temp
*.swp
*.swo
*~
# IDE
.vscode/
.idea/
*.sublime-project
*.sublime-workspace
# OS
.DS_Store
Thumbs.db
# Service specific
/tmp/ktv-synth-outputs/
*.mp4
*.wav
*.mkv

255
README.md
View File

@ -1,255 +1,2 @@
# KTV Synth Service
# ktv-synth-service
KTV/MTV video synthesis service using FFmpeg. Creates karaoke videos with dual audio tracks (accompaniment + original) and synchronized ASS subtitles.
## Overview
This service processes video clips, audio tracks, and subtitles to produce:
- **MTV (Music Television)**: Single audio track with original vocals and subtitles
- **KTV (Karaoke Television)**: Dual audio tracks - accompaniment (default) and original vocals
## Architecture
- **Framework**: ahserver + longtasks + Redis
- **Port**: 9084
- **Queue**: `ktv_synth`
- **Worker**: FFmpeg subprocess (no GPU required)
## Features
- Two-step FFmpeg synthesis pipeline
- ASS subtitle rendering with karaoke effects
- Dual audio track support with proper metadata
- Configurable video looping for scene clips
- 1920x1080 output resolution with Lanczos scaling
- Automatic duration calculation
## Installation
### Prerequisites
- Python 3.8+
- FFmpeg with libx264 and AAC support
- Redis server
### Setup
```bash
# Clone repository
cd /data/ymq/ktv-synth-service
# Ensure FFmpeg is installed
ffmpeg -version
# Ensure Redis is running
redis-cli ping
```
## Usage
### Starting the Service
```bash
./start.sh
```
The service will start on port 9084 and begin processing tasks from the Redis queue.
### Stopping the Service
```bash
./stop.sh
```
### Health Check
Visit `http://localhost:9084/app/health.dspy` or check the service status.
## API
### Task Payload
Submit tasks to the Redis queue `ktv_synth`:
```json
{
"task_type": "synthesize",
"video_files": [
"/path/to/scene1.mp4",
"/path/to/scene2.mp4",
"/path/to/scene3.mp4"
],
"original_audio": "/path/to/original.wav",
"accompaniment": "/path/to/no_vocals.wav",
"subtitle_path": "/path/to/subtitles.ass",
"output_dir": "/tmp/ktv-synth-outputs",
"title": "SongName",
"duration": 240.5,
"loops": 3,
"output_modes": ["mtv", "ktv"]
}
```
### Parameters
- `video_files` (required): List of video file paths (scene clips to loop)
- `original_audio` (required): Path to original full audio with vocals
- `accompaniment` (required for KTV): Path to accompaniment track (no vocals)
- `subtitle_path` (required): Path to ASS subtitle file
- `output_dir` (optional): Output directory (default: `/tmp/ktv-synth-outputs`)
- `title` (optional): Song title for output naming (default: `output`)
- `duration` (optional): Target duration in seconds (auto-calculated if not provided)
- `loops` (optional): Number of video loops (auto-calculated if not provided)
- `output_modes` (optional): List of outputs to generate: `["mtv"]`, `["ktv"]`, or `["mtv", "ktv"]`
### Response
```json
{
"mtv_path": "/tmp/ktv-synth-outputs/SongName_MTV.mp4",
"ktv_path": "/tmp/ktv-synth-outputs/SongName_KTV.mp4",
"mtv_size_mb": 125.45,
"ktv_size_mb": 145.67,
"duration": 240.5
}
```
## Technical Details
### Two-Step Synthesis Process
#### Step 1: Create Silent Looped Video Track
Concatenates and loops scene clips to match target duration:
```bash
ffmpeg -y -f concat -safe 0 -stream_loop {loops} -i {concat_list} \
-t {duration} -an -c:v libx264 -preset fast -crf 23 {temp_video}
```
#### Step 2a: MTV Synthesis (Single Track)
Combines video with original audio and ASS subtitles:
```bash
ffmpeg -y -i {temp_video} -i {original_audio} \
-map 0:v -map 1:a \
-vf "ass={subtitle_path},scale=1920:1080:flags=lanczos" \
-c:v libx264 -preset fast -crf 23 \
-c:a aac -b:a 192k \
{mtv_output}
```
#### Step 2b: KTV Synthesis (Dual Track)
Creates dual audio tracks with accompaniment as default:
```bash
ffmpeg -y -i {temp_video} -i {accompaniment} -i {original_audio} \
-map 0:v -map 1:a -map 2:a \
-vf "ass={subtitle_path},scale=1920:1080:flags=lanczos" \
-c:v libx264 -preset fast -crf 23 \
-c:a:0 aac -b:a:0 192k -metadata:s:a:0 handler_name="伴奏(Accompaniment)" \
-c:a:1 aac -b:a:1 192k -metadata:s:a:1 handler_name="原唱(Original)" \
-disposition:a:0 default -disposition:a:1 0 \
{ktv_output}
```
### Video Encoding Settings
- **Codec**: H.264 (libx264)
- **Preset**: fast
- **CRF**: 23 (balanced quality/size)
- **Resolution**: 1920x1080
- **Scaling**: Lanczos (high quality)
### Audio Encoding Settings
- **Codec**: AAC
- **Bitrate**: 192 kbps
- **Tracks**: 1 (MTV) or 2 (KTV)
### KTV Audio Track Metadata
- **Track 0**: Accompaniment (default playback)
- Handler: "伴奏(Accompaniment)"
- Disposition: default
- **Track 1**: Original with vocals
- Handler: "原唱(Original)"
- Disposition: 0 (not default)
## Configuration
Edit `conf/config.json`:
```json
{
"port": 9084,
"queue": "ktv_synth",
"filesroot": "/tmp/ktv-synth-outputs",
"redis_url": "redis://127.0.0.1:6379",
"worker_cnt": 1,
"stuck_seconds": 1800,
"max_age_hours": 24
}
```
## Troubleshooting
### FFmpeg Errors
Check FFmpeg installation and codec support:
```bash
ffmpeg -codecs | grep libx264
ffmpeg -codecs | grep aac
```
### Redis Connection
Verify Redis is running:
```bash
redis-cli ping
```
### Permission Issues
Ensure the service has write access to output directories:
```bash
chmod 755 /tmp/ktv-synth-outputs
```
### High Memory Usage
Reduce worker count in `conf/config.json`:
```json
{
"worker_cnt": 1
}
```
## Performance
- **MTV Generation**: ~2-3x real-time (240s video in ~80-120s)
- **KTV Generation**: ~2-3x real-time
- **Concurrent Tasks**: Limited by `worker_cnt` (default: 1)
- **Memory**: ~500MB-1GB per worker (depends on video resolution)
## Integration
This service integrates with:
- **demucs-service**: Audio source separation (provides accompaniment tracks)
- **whisper-service**: Subtitle generation (provides ASS files)
- **wan22-service**: Video generation (provides scene clips)
## License
Internal use only.
## Support
For issues or questions, contact the development team.

33
ah.py
View File

@ -1,33 +0,0 @@
import os
from ahserver.webapp import webapp
from ahserver.serverenv import ServerEnv
from ahserver.configuredServer import add_startup
from longtasks.longtasks import LongTasks, schedule_once
from appPublic.log import debug
class KTVSynthTasks(LongTasks):
def __init__(self, *args, **kwargs):
super().__init__(*args, **kwargs)
async def process_task(self, payload, workid=None):
import json
if isinstance(payload, str): payload = json.loads(payload)
task_type = payload.get('task_type', '')
if task_type == 'synthesize':
from workers.synthesize import run_synthesize
return await run_synthesize(self, payload)
raise ValueError(f'Unknown task_type: {task_type}')
async def on_app_built(app):
env = ServerEnv()
lt = env.longtasks
if lt:
schedule_once(0.1, lt.run)
debug(f'KTV synth longtasks worker started')
def init():
env = ServerEnv()
env.longtasks = KTVSynthTasks('redis://127.0.0.1:6379', 'ktv_synth', worker_cnt=1, stuck_seconds=1800, max_age_hours=24)
add_startup(on_app_built)
if __name__ == '__main__':
webapp(init)

1
app/__init__.py
View File

@ -1 +0,0 @@
# KTV Synth Service App

13
app/api/status/index.dspy
View File

@ -1,13 +0,0 @@
# -*- coding:utf-8 -*-
# GET /api/status - KTV合成服务状态
import subprocess
import json
result = {
'service': 'ktv-video-synthesis',
'components': ['ffmpeg', 'ass-subtitle', 'video-concat'],
'gpu_required': False
}
return json.dumps(result)

66
app/api/submit/index.dspy
View File

@ -1,66 +0,0 @@
# -*- coding:utf-8 -*-
# POST /api/submit - 提交KTV视频合成任务
import json
import uuid
from ahserver.serverenv import ServerEnv
method = request.method
if method == 'POST':
# 必需参数
video_clips = params_kw.get('video_clips', [])
audio_path = params_kw.get('audio_path', '')
subtitle_path = params_kw.get('subtitle_path', '')
if not video_clips or not isinstance(video_clips, list):
return json.dumps({'error': 'video_clips (list) is required'}, ensure_ascii=False)
if not audio_path:
return json.dumps({'error': 'audio_path is required'}, ensure_ascii=False)
if not subtitle_path:
return json.dumps({'error': 'subtitle_path (ASS file) is required'}, ensure_ascii=False)
task_id = params_kw.get('task_id', str(uuid.uuid4()).replace("-", "")[:12])
output_format = params_kw.get('output_format', 'mp4')
resolution = params_kw.get('resolution', '1920x1080')
payload = {
'task_type': 'synthesize',
'task_id': task_id,
'video_clips': video_clips,
'audio_path': audio_path,
'subtitle_path': subtitle_path,
'output_format': output_format,
'resolution': resolution
}
env = ServerEnv()
longtasks = env.longtasks
if longtasks is None:
return json.dumps({'error': 'service not ready'}, ensure_ascii=False)
result = await longtasks.submit_task(payload)
real_task_id = result.get('task_id', str(result)) if isinstance(result, dict) else str(result)
return json.dumps({
'task_id': real_task_id,
'status': 'queued',
'video_clips_count': len(video_clips),
'audio_path': audio_path,
'subtitle_path': subtitle_path,
'message': 'task submitted',
'check_url': f'/api/task?task_id={real_task_id}'
}, ensure_ascii=False)
else:
return json.dumps({
'usage': 'POST with JSON body',
'params': {
'video_clips': 'list (required, server paths to video segments)',
'audio_path': 'string (required, server path to audio file)',
'subtitle_path': 'string (required, server path to ASS subtitle file)',
'output_format': 'string (default mp4)',
'resolution': 'string (default 1920x1080)',
'task_id': 'string (optional, auto-generated)',
}
}, ensure_ascii=False)

17
app/api/task/index.dspy
View File

@ -1,17 +0,0 @@
# -*- coding:utf-8 -*-
# GET /api/task?task_id=xxx - 查询任务状态
import json
from ahserver.serverenv import ServerEnv
task_id = params_kw.get('task_id', '')
if not task_id:
return json.dumps({'error': 'task_id is required'}, ensure_ascii=False)
env = ServerEnv()
longtasks = env.longtasks
if longtasks is None:
return json.dumps({'error': 'service not ready'}, ensure_ascii=False)
status = await longtasks.get_status(task_id)
return json.dumps(status)

3
app/health.dspy
View File

@ -1,3 +0,0 @@
import json
result = {"status": "ok", "service": "$svc"}
print(json.dumps(result))

150
build.sh
View File

@ -1,150 +0,0 @@
#!/bin/bash
# 一键部署脚本模板
# 用法: ./build.sh [deploy|update|stop|status]
set -e
SERVICE_NAME="ktv-synth-service"
GIT_REPO="git@git.opencomputing.cn:yumoqing/ktv-synth-service.git"
SERVICE_PORT=9084
DEPLOY_DIR="/data/ymq/$SERVICE_NAME"
VENV_PATH="/data/ymq/wan22-service/py3"
GPU_ID="CPU"
# 颜色输出
RED='\033[0;31m'
GREEN='\033[0;32m'
YELLOW='\033[1;33m'
NC='\033[0m'
log_info() { echo -e "${GREEN}[INFO]${NC} $1"; }
log_warn() { echo -e "${YELLOW}[WARN]${NC} $1"; }
log_error() { echo -e "${RED}[ERROR]${NC} $1"; }
check_deps() {
command -v git >/dev/null || { log_error "git not found"; exit 1; }
[ -f "$VENV_PATH/bin/python" ] || { log_error "Python venv not found: $VENV_PATH"; exit 1; }
}
deploy() {
log_info "Deploying $SERVICE_NAME..."
# 检查依赖
check_deps
# 克隆或更新代码
if [ -d "$DEPLOY_DIR/.git" ]; then
log_info "Updating existing deployment..."
cd "$DEPLOY_DIR"
git fetch origin
git reset --hard origin/master
else
log_info "Cloning repository..."
cd /data/ymq
git clone "$GIT_REPO" "$SERVICE_NAME"
cd "$DEPLOY_DIR"
fi
# 创建必要目录
mkdir -p "$DEPLOY_DIR/app/api/status"
mkdir -p "$DEPLOY_DIR/app/api/submit"
mkdir -p "$DEPLOY_DIR/app/api/task"
# 设置权限
chmod +x start.sh stop.sh 2>/dev/null || true
# 启动服务
start_service
}
start_service() {
log_info "Starting $SERVICE_NAME on port $SERVICE_PORT..."
# 停止旧进程
if [ -f stop.sh ]; then
bash stop.sh 2>/dev/null || true
sleep 2
fi
# 启动新进程
bash start.sh
# 等待启动
sleep 3
# 验证
if ss -tlnp | grep -q ":$SERVICE_PORT "; then
log_info "✓ Service started successfully"
verify_api
else
log_error "✗ Service failed to start"
log_error "Check logs: $DEPLOY_DIR/nohup.out"
exit 1
fi
}
verify_api() {
log_info "Verifying API endpoints..."
# 检查 status endpoint
if curl -s "http://127.0.0.1:$SERVICE_PORT/api/status" | grep -q "service"; then
log_info "✓ /api/status OK"
else
log_warn "✗ /api/status failed"
fi
}
stop_service() {
log_info "Stopping $SERVICE_NAME..."
if [ -f "$DEPLOY_DIR/stop.sh" ]; then
cd "$DEPLOY_DIR"
bash stop.sh
log_info "✓ Service stopped"
else
log_warn "stop.sh not found"
fi
}
show_status() {
echo "=== $SERVICE_NAME Status ==="
echo "Port: $SERVICE_PORT"
echo "Deploy Dir: $DEPLOY_DIR"
echo ""
# 检查进程
if ss -tlnp | grep -q ":$SERVICE_PORT "; then
echo -e "Status: ${GREEN}RUNNING${NC}"
PID=$(ss -tlnp | grep ":$SERVICE_PORT " | grep -oP 'pid=\K[0-9]+')
echo "PID: $PID"
else
echo -e "Status: ${RED}STOPPED${NC}"
fi
echo ""
# 检查 API
echo "API Endpoints:"
curl -s "http://127.0.0.1:$SERVICE_PORT/api/status" 2>/dev/null | python3 -m json.tool 2>/dev/null || echo " (not responding)"
}
# 主入口
case "${1:-deploy}" in
deploy|install)
deploy
;;
update|upgrade)
deploy
;;
stop)
stop_service
;;
start)
start_service
;;
status)
show_status
;;
*)
echo "Usage: $0 {deploy|update|stop|start|status}"
exit 1
;;
esac

38
conf/config.json
View File

@ -1,38 +0,0 @@
{
"password_key": "KTVSynthService2026Key",
"databases": {},
"session_redis": {
"host": "127.0.0.1",
"port": 6379,
"db": 1
},
"website": {
"paths": [
[
"$[workdir]$/app",
""
]
],
"host": "0.0.0.0",
"port": 9084,
"coding": "utf-8",
"indexes": [
"index.html",
"index.dspy"
],
"processors": [
[
".dspy",
"dspy"
]
],
"startswiths": [
{
"leading": "/idfile",
"registerfunction": "idfile"
}
]
},
"hot_reload": false,
"filesroot": "/tmp/ktv-synth-outputs"
}

5
start.sh
View File

@ -1,5 +0,0 @@
#!/bin/bash
cd /data/ymq/ktv-synth-service
export PYTHONPATH=/data/ymq/ktv-synth-service
nohup /data/ymq/wan22-service/py3/bin/python ah.py > nohup.out 2>&1 &
echo "ktv-synth-service started, PID: $!"

31
stop.sh
View File

@ -1,31 +0,0 @@
#!/bin/bash
# Stop ktv-synth-service
SERVICE_NAME="ah.py"
PID_FILE="nohup.out"
# Find process
PID=$(ps aux | grep "[p]ython.*$SERVICE_NAME" | awk '{print $2}')
if [ -z "$PID" ]; then
echo "ktv-synth-service is not running"
exit 0
fi
echo "Stopping ktv-synth-service (PID: $PID)..."
kill $PID
# Wait for process to stop
for i in {1..10}; do
if ! ps -p $PID > /dev/null 2>&1; then
echo "ktv-synth-service stopped successfully"
exit 0
fi
sleep 1
done
# Force kill if still running
echo "Force killing ktv-synth-service..."
kill -9 $PID
echo "ktv-synth-service force stopped"

1
workers/__init__.py
View File

@ -1 +0,0 @@
# KTV Synth Service Workers

167
workers/synthesize.py
View File

@ -1,167 +0,0 @@
import os
import json
import tempfile
import subprocess
import asyncio
from pathlib import Path
from appPublic.log import debug, error
async def run_synthesize(task_instance, payload):
"""
Run KTV/MTV video synthesis using two-step ffmpeg process.
Step 1: Create silent looped video track from scene clips
Step 2a: MTV - single track with original audio + ASS subtitles
Step 2b: KTV - dual track with accompaniment (default) + original audio
"""
try:
# Extract parameters from payload
video_files = payload.get('video_files', [])
original_audio = payload.get('original_audio')
accompaniment = payload.get('accompaniment')
subtitle_path = payload.get('subtitle_path')
output_dir = payload.get('output_dir', '/tmp/ktv-synth-outputs')
title = payload.get('title', 'output')
duration = payload.get('duration')
loops = payload.get('loops')
output_modes = payload.get('output_modes', ['mtv', 'ktv'])
# Validate required parameters
if not video_files:
raise ValueError('video_files is required')
if not original_audio:
raise ValueError('original_audio is required')
if not subtitle_path:
raise ValueError('subtitle_path is required')
# Create output directory
os.makedirs(output_dir, exist_ok=True)
# Create temporary directory for intermediate files
with tempfile.TemporaryDirectory() as temp_dir:
temp_video = os.path.join(temp_dir, 'temp_video.mp4')
concat_list = os.path.join(temp_dir, 'concat_list.txt')
# Create concat list for ffmpeg
with open(concat_list, 'w') as f:
for video_file in video_files:
f.write(f"file '{video_file}'\n")
# Calculate loops if not provided
if loops is None:
# Get duration of first video file to estimate total duration
probe_cmd = [
'ffprobe', '-v', 'error', '-show_entries', 'format=duration',
'-of', 'default=noprint_wrappers=1:nokey=1', video_files[0]
]
result = subprocess.run(probe_cmd, capture_output=True, text=True)
clip_duration = float(result.stdout.strip())
# If duration not provided, use sum of all clips
if duration is None:
total_duration = 0
for vf in video_files:
probe_cmd = [
'ffprobe', '-v', 'error', '-show_entries', 'format=duration',
'-of', 'default=noprint_wrappers=1:nokey=1', vf
]
result = subprocess.run(probe_cmd, capture_output=True, text=True)
total_duration += float(result.stdout.strip())
duration = total_duration
loops = int((duration / (clip_duration * len(video_files))) + 1)
debug(f'Starting synthesis: title={title}, duration={duration}, loops={loops}, modes={output_modes}')
# Step 1: Create silent looped video track
debug('Step 1: Creating silent looped video track')
cmd_step1 = [
'ffmpeg', '-y', '-f', 'concat', '-safe', '0',
'-stream_loop', str(loops),
'-i', concat_list,
'-t', str(duration),
'-an', # No audio
'-c:v', 'libx264', '-preset', 'fast', '-crf', '23',
temp_video
]
debug(f'Running: {" ".join(cmd_step1)}')
result = subprocess.run(cmd_step1, capture_output=True, text=True)
if result.returncode != 0:
error(f'FFmpeg step 1 failed: {result.stderr}')
raise RuntimeError(f'FFmpeg step 1 failed: {result.stderr}')
result_dict = {'duration': duration}
# Step 2a: MTV synthesis (if requested)
if 'mtv' in output_modes:
debug('Step 2a: Creating MTV (single track)')
mtv_output = os.path.join(output_dir, f'{title}_MTV.mp4')
cmd_mtv = [
'ffmpeg', '-y',
'-i', temp_video,
'-i', original_audio,
'-map', '0:v',
'-map', '1:a',
'-vf', f'ass={subtitle_path},scale=1920:1080:flags=lanczos',
'-c:v', 'libx264', '-preset', 'fast', '-crf', '23',
'-c:a', 'aac', '-b:a', '192k',
mtv_output
]
debug(f'Running: {" ".join(cmd_mtv)}')
result = subprocess.run(cmd_mtv, capture_output=True, text=True)
if result.returncode != 0:
error(f'FFmpeg MTV synthesis failed: {result.stderr}')
raise RuntimeError(f'FFmpeg MTV synthesis failed: {result.stderr}')
mtv_size = os.path.getsize(mtv_output) / (1024 * 1024) # Size in MB
result_dict['mtv_path'] = mtv_output
result_dict['mtv_size_mb'] = round(mtv_size, 2)
debug(f'MTV created: {mtv_output} ({mtv_size:.2f} MB)')
# Step 2b: KTV synthesis (if requested)
if 'ktv' in output_modes:
if not accompaniment:
raise ValueError('accompaniment is required for KTV output')
debug('Step 2b: Creating KTV (dual track)')
ktv_output = os.path.join(output_dir, f'{title}_KTV.mp4')
cmd_ktv = [
'ffmpeg', '-y',
'-i', temp_video,
'-i', accompaniment,
'-i', original_audio,
'-map', '0:v',
'-map', '1:a', # Accompaniment
'-map', '2:a', # Original
'-vf', f'ass={subtitle_path},scale=1920:1080:flags=lanczos',
'-c:v', 'libx264', '-preset', 'fast', '-crf', '23',
'-c:a:0', 'aac', '-b:a:0', '192k',
'-metadata:s:a:0', 'handler_name=伴奏(Accompaniment)',
'-c:a:1', 'aac', '-b:a:1', '192k',
'-metadata:s:a:1', 'handler_name=原唱(Original)',
'-disposition:a:0', 'default',
'-disposition:a:1', '0',
ktv_output
]
debug(f'Running: {" ".join(cmd_ktv)}')
result = subprocess.run(cmd_ktv, capture_output=True, text=True)
if result.returncode != 0:
error(f'FFmpeg KTV synthesis failed: {result.stderr}')
raise RuntimeError(f'FFmpeg KTV synthesis failed: {result.stderr}')
ktv_size = os.path.getsize(ktv_output) / (1024 * 1024) # Size in MB
result_dict['ktv_path'] = ktv_output
result_dict['ktv_size_mb'] = round(ktv_size, 2)
debug(f'KTV created: {ktv_output} ({ktv_size:.2f} MB)')
debug(f'Synthesis completed successfully: {json.dumps(result_dict)}')
return result_dict
except Exception as e:
error(f'Synthesis failed: {str(e)}')
raise