AIEC-new/AIEC-RAG/DEPLOYMENT_GUIDE.md

# AIEC-RAG 部署指南

## 目录

1. [部署架构](#部署架构)
2. [单机部署](#单机部署)
3. [Docker部署](#docker部署)
4. [生产环境部署](#生产环境部署)
5. [性能调优](#性能调优)
6. [监控配置](#监控配置)
7. [备份恢复](#备份恢复)

## 部署架构

### 推荐架构

```
                    [负载均衡器]
                         |
            ┌────────────┼────────────┐
            ↓            ↓            ↓
      [AIEC-RAG-1] [AIEC-RAG-2] [AIEC-RAG-3]
            ↓            ↓            ↓
            └────────────┼────────────┘
                         ↓
                  [Elasticsearch集群]
                         ↓
                   [向量数据库]
```

### 最小配置要求

| 组件 | CPU | 内存 | 存储 | 说明 |
|-----|-----|------|------|------|
| API服务 | 4核 | 8GB | 50GB | 单实例最小配置 |
| Elasticsearch | 4核 | 16GB | 200GB | 推荐使用SSD |
| 整体系统 | 8核 | 32GB | 500GB | 生产环境推荐 |

## 单机部署

### 1. 系统准备

```bash
# Ubuntu/Debian
sudo apt update
sudo apt install -y python3.8 python3-pip git curl wget

# CentOS/RHEL
sudo yum update -y
sudo yum install -y python38 python38-pip git curl wget
```

### 2. 安装Elasticsearch

```bash
# 下载并安装Elasticsearch 8.x
wget https://artifacts.elastic.co/downloads/elasticsearch/elasticsearch-8.11.0-linux-x86_64.tar.gz
tar -xzf elasticsearch-8.11.0-linux-x86_64.tar.gz
cd elasticsearch-8.11.0

# 配置Elasticsearch
cat >> config/elasticsearch.yml << EOF
network.host: 0.0.0.0
discovery.type: single-node
xpack.security.enabled: true
xpack.security.authc.api_key.enabled: true
EOF

# 启动Elasticsearch
./bin/elasticsearch -d
```

### 3. 部署AIEC-RAG

```bash
# 克隆项目
git clone <repository_url>
cd AIEC-RAG

# 创建虚拟环境
python3 -m venv venv
source venv/bin/activate

# 安装依赖
pip install -r requirements.txt

# 配置环境变量
cp .env.example .env
# 编辑.env文件，填入实际配置

# 启动服务
python rag_api_server_production.py
```

### 4. 设置系统服务

创建 `/etc/systemd/system/aiec-rag.service`:

```ini
[Unit]
Description=AIEC-RAG Service
After=network.target elasticsearch.service

[Service]
Type=simple
User=aiec
WorkingDirectory=/opt/AIEC-RAG
Environment="PATH=/opt/AIEC-RAG/venv/bin"
ExecStart=/opt/AIEC-RAG/venv/bin/python /opt/AIEC-RAG/rag_api_server_production.py
Restart=always
RestartSec=10

[Install]
WantedBy=multi-user.target
```

启用服务：

```bash
sudo systemctl daemon-reload
sudo systemctl enable aiec-rag
sudo systemctl start aiec-rag
sudo systemctl status aiec-rag
```

## Docker部署

### 1. 使用预构建镜像

```bash
# 拉取镜像（如果有私有仓库）
docker pull your-registry/aiec-rag:latest

# 或构建本地镜像
docker build -t aiec-rag:latest .
```

### 2. Docker Compose部署

创建 `docker-compose.yml`:

```yaml
version: '3.8'

services:
  elasticsearch:
    image: docker.elastic.co/elasticsearch/elasticsearch:8.11.0
    container_name: aiec-elasticsearch
    environment:
      - discovery.type=single-node
      - "ES_JAVA_OPTS=-Xms2g -Xmx2g"
      - xpack.security.enabled=true
      - ELASTIC_PASSWORD=your_password
    volumes:
      - es_data:/usr/share/elasticsearch/data
    ports:
      - "9200:9200"
    networks:
      - aiec_network
    healthcheck:
      test: ["CMD", "curl", "-f", "http://localhost:9200"]
      interval: 30s
      timeout: 10s
      retries: 5

  aiec-rag:
    build: .
    container_name: aiec-rag
    depends_on:
      elasticsearch:
        condition: service_healthy
    environment:
      - ELASTICSEARCH_HOST=http://elasticsearch:9200
      - ELASTICSEARCH_USERNAME=elastic
      - ELASTICSEARCH_PASSWORD=your_password
    env_file:
      - .env
    ports:
      - "8100:8100"
    volumes:
      - ./rag_config_production.yaml:/app/rag_config_production.yaml
      - ./api_outputs:/app/api_outputs
    networks:
      - aiec_network
    restart: unless-stopped

volumes:
  es_data:
    driver: local

networks:
  aiec_network:
    driver: bridge
```

启动服务：

```bash
docker-compose up -d
docker-compose logs -f
```

### 3. Kubernetes部署

创建 `k8s-deployment.yaml`:

```yaml
apiVersion: apps/v1
kind: Deployment
metadata:
  name: aiec-rag
  labels:
    app: aiec-rag
spec:
  replicas: 3
  selector:
    matchLabels:
      app: aiec-rag
  template:
    metadata:
      labels:
        app: aiec-rag
    spec:
      containers:
      - name: aiec-rag
        image: your-registry/aiec-rag:latest
        ports:
        - containerPort: 8100
        env:
        - name: ELASTICSEARCH_HOST
          value: "http://elasticsearch-service:9200"
        envFrom:
        - secretRef:
            name: aiec-secrets
        resources:
          requests:
            memory: "4Gi"
            cpu: "2"
          limits:
            memory: "8Gi"
            cpu: "4"
        livenessProbe:
          httpGet:
            path: /health
            port: 8100
          initialDelaySeconds: 30
          periodSeconds: 10
        readinessProbe:
          httpGet:
            path: /health
            port: 8100
          initialDelaySeconds: 5
          periodSeconds: 5
---
apiVersion: v1
kind: Service
metadata:
  name: aiec-rag-service
spec:
  selector:
    app: aiec-rag
  ports:
    - protocol: TCP
      port: 80
      targetPort: 8100
  type: LoadBalancer
```

部署到Kubernetes:

```bash
# 创建密钥
kubectl create secret generic aiec-secrets --from-env-file=.env

# 部署应用
kubectl apply -f k8s-deployment.yaml

# 查看状态
kubectl get pods
kubectl get services
```

## 生产环境部署

### 1. 负载均衡配置

使用Nginx作为负载均衡器：

```nginx
upstream aiec_backend {
    least_conn;
    server 10.0.1.10:8100 weight=1 max_fails=3 fail_timeout=30s;
    server 10.0.1.11:8100 weight=1 max_fails=3 fail_timeout=30s;
    server 10.0.1.12:8100 weight=1 max_fails=3 fail_timeout=30s;
}

server {
    listen 80;
    server_name api.aiec-rag.com;

    location / {
        proxy_pass http://aiec_backend;
        proxy_http_version 1.1;
        proxy_set_header Upgrade $http_upgrade;
        proxy_set_header Connection 'upgrade';
        proxy_set_header Host $host;
        proxy_cache_bypass $http_upgrade;
        proxy_set_header X-Real-IP $remote_addr;
        proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
        proxy_set_header X-Forwarded-Proto $scheme;
        
        # 超时设置
        proxy_connect_timeout 60s;
        proxy_send_timeout 120s;
        proxy_read_timeout 120s;
    }
    
    # 健康检查端点
    location /health {
        proxy_pass http://aiec_backend/health;
    }
}
```

### 2. SSL/TLS配置

```nginx
server {
    listen 443 ssl http2;
    server_name api.aiec-rag.com;

    ssl_certificate /etc/nginx/ssl/aiec-rag.crt;
    ssl_certificate_key /etc/nginx/ssl/aiec-rag.key;
    
    ssl_protocols TLSv1.2 TLSv1.3;
    ssl_ciphers HIGH:!aNULL:!MD5;
    ssl_prefer_server_ciphers on;
    
    # ... 其他配置同上
}
```

### 3. 数据库优化

Elasticsearch优化配置：

```yaml
# elasticsearch.yml
cluster.name: aiec-rag-cluster
node.name: node-1

# 内存设置
bootstrap.memory_lock: true

# 线程池
thread_pool:
  write:
    size: 8
    queue_size: 1000
  search:
    size: 16
    queue_size: 1000

# 索引设置
index:
  number_of_shards: 3
  number_of_replicas: 1
  refresh_interval: 30s
```

## 性能调优

### 1. Python应用优化

```python
# 使用Gunicorn作为WSGI服务器（Linux）
gunicorn -w 4 -k uvicorn.workers.UvicornWorker \
    --bind 0.0.0.0:8100 \
    --timeout 120 \
    --keep-alive 5 \
    --max-requests 1000 \
    --max-requests-jitter 50 \
    rag_api_server_production:app
```

### 2. 系统参数优化

```bash
# /etc/sysctl.conf
net.ipv4.tcp_fin_timeout = 30
net.ipv4.tcp_tw_reuse = 1
net.ipv4.tcp_tw_recycle = 1
net.ipv4.tcp_max_syn_backlog = 8192
net.ipv4.tcp_max_tw_buckets = 10000
net.core.somaxconn = 65535
net.core.netdev_max_backlog = 65535

# 应用配置
sudo sysctl -p
```

### 3. 缓存策略

配置Redis缓存：

```python
# 在代码中添加缓存支持
import redis
from functools import lru_cache

redis_client = redis.Redis(
    host='localhost',
    port=6379,
    decode_responses=True,
    max_connections=50
)

@lru_cache(maxsize=128)
def get_cached_embedding(text: str):
    # 缓存嵌入向量
    pass
```

## 监控配置

### 1. Prometheus监控

```yaml
# prometheus.yml
scrape_configs:
  - job_name: 'aiec-rag'
    static_configs:
      - targets: ['localhost:8100']
    metrics_path: '/metrics'
    scrape_interval: 15s
```

### 2. 日志管理

配置日志轮转：

```bash
# /etc/logrotate.d/aiec-rag
/opt/AIEC-RAG/logs/*.log {
    daily
    rotate 30
    compress
    delaycompress
    missingok
    notifempty
    create 644 aiec aiec
    sharedscripts
    postrotate
        systemctl reload aiec-rag
    endscript
}
```

### 3. 告警配置

```yaml
# alerting_rules.yml
groups:
- name: aiec_alerts
  rules:
  - alert: HighResponseTime
    expr: http_request_duration_seconds{quantile="0.99"} > 5
    for: 5m
    labels:
      severity: warning
    annotations:
      summary: "High response time on {{ $labels.instance }}"
      
  - alert: ServiceDown
    expr: up{job="aiec-rag"} == 0
    for: 1m
    labels:
      severity: critical
    annotations:
      summary: "AIEC-RAG service is down"
```

## 备份恢复

### 1. 数据备份

```bash
#!/bin/bash
# backup.sh
DATE=$(date +%Y%m%d_%H%M%S)
BACKUP_DIR="/backup/aiec-rag"

# 备份Elasticsearch数据
curl -X PUT "localhost:9200/_snapshot/backup_repo" -H 'Content-Type: application/json' -d'
{
  "type": "fs",
  "settings": {
    "location": "'$BACKUP_DIR'/elasticsearch"
  }
}'

curl -X PUT "localhost:9200/_snapshot/backup_repo/snapshot_$DATE?wait_for_completion=true"

# 备份配置文件
tar -czf $BACKUP_DIR/config_$DATE.tar.gz \
    /opt/AIEC-RAG/.env \
    /opt/AIEC-RAG/rag_config_production.yaml

echo "Backup completed: $DATE"
```

### 2. 恢复流程

```bash
#!/bin/bash
# restore.sh
SNAPSHOT_NAME=$1

# 恢复Elasticsearch数据
curl -X POST "localhost:9200/_snapshot/backup_repo/$SNAPSHOT_NAME/_restore"

# 恢复配置文件
tar -xzf /backup/aiec-rag/config_latest.tar.gz -C /

# 重启服务
systemctl restart aiec-rag

echo "Restore completed from: $SNAPSHOT_NAME"
```

## 故障处理

### 常见问题处理

1. **服务无响应**
   ```bash
   # 检查服务状态
   systemctl status aiec-rag
   # 查看日志
   journalctl -u aiec-rag -n 100
   # 重启服务
   systemctl restart aiec-rag
   ```

2. **Elasticsearch连接失败**
   ```bash
   # 检查ES状态
   curl -X GET "localhost:9200/_cluster/health?pretty"
   # 检查网络连接
   telnet localhost 9200
   ```

3. **内存溢出**
   ```bash
   # 增加内存限制
   export PYTHONUNBUFFERED=1
   export OMP_NUM_THREADS=4
   ```

## 安全建议

1. **API密钥管理**
   - 使用密钥管理服务（如HashiCorp Vault）
   - 定期轮换API密钥
   - 不要在代码中硬编码密钥

2. **网络安全**
   - 使用防火墙限制访问
   - 配置SSL/TLS加密
   - 实施速率限制

3. **数据安全**
   - 加密敏感数据
   - 定期备份
   - 实施访问控制

---

*更多部署问题，请参考项目Wiki或联系技术支持。*