icecheng/freeleaps-ops
1
0
Fork 0
Code Issues Pull Requests Packages Projects Releases Wiki Activity

chore(i18n): replace all Chinese content with English in gitea-webhook-ambassador-python

Browse Source
This commit is contained in:
Nicolas 2025-07-22 18:54:54 +08:00
parent c259460f9c
commit 9dbee47706
34 changed files with 1219 additions and 1511 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files

170
apps/gitea-webhook-ambassador-python/README.md
View File

@ -1,110 +1,110 @@
# Gitea Webhook Ambassador (Python) # Gitea Webhook Ambassador (Python)
一个高性能的 Python webhook 服务，用于连接 Gitea 和 Jenkins，支持智能分发、高并发处理和防抖策略。 A high-performance Python webhook service for connecting Gitea and Jenkins, supporting intelligent dispatch, high concurrency processing, and deduplication strategies.
## 🚀 新特性 ## 🚀 Features
### 1. 智能分发策略 ### 1. Intelligent Dispatch Strategy
- **dev 分支** → 触发 alpha 环境构建 - **dev branches** → Trigger alpha environment build
- **prod 分支** → 触发生产环境构建 - **prod branches** → Trigger production environment build
- **其他分支** → 可配置的默认策略 - **other branches** → Configurable default strategy
### 2. 高并发处理 ### 2. High Concurrency Processing
- **异步任务队列**: 使用 Celery + Redis 处理高并发 - **Asynchronous Task Queue**: Use Celery + Redis for high concurrency
- **任务排队机制**: 防止构建丢失，确保任务按序执行 - **Task Queueing Mechanism**: Prevent build loss, ensure tasks are executed in order
- **负载均衡**: 支持多 worker 实例 - **Load Balancing**: Support multiple worker instances
### 3. 防抖策略 ### 3. Deduplication Strategy
- **基于 commit hash + 分支的去重**: 防止重复触发 - **Deduplication based on commit hash + branch**: Prevent repeated triggers
- **时间窗口防抖**: 在指定时间窗口内的相同提交只触发一次 - **Time Window Deduplication**: Only trigger once for the same commit within a specified time window
- **智能去重**: 支持配置去重策略 - **Intelligent Deduplication**: Support configurable deduplication strategies
## 🏗️ 架构设计 ## 🏗️ Architecture Design
``` ```
Gitea Webhook → FastAPI → Celery Queue → Jenkins Workers Gitea Webhook → FastAPI → Celery Queue → Jenkins Workers
↓ ↓ ↓ ↓ ↓ ↓ ↓ ↓
验证签名 路由分发 任务排队 并发执行 Signature Verification Routing Dispatch Task Queueing Concurrent Execution
↓ ↓ ↓ ↓ ↓ ↓ ↓ ↓
防抖检查 环境判断 持久化存储 状态反馈 Deduplication Environment Judgment Persistent Storage Status Feedback
``` ```
## 📁 项目结构 ## 📁 Project Structure
``` ```
gitea-webhook-ambassador-python/ gitea-webhook-ambassador-python/
├── app/ ├── app/
│ ├── __init__.py │ ├── __init__.py
│ ├── main.py # FastAPI 应用入口 │ ├── main.py # FastAPI application entry
│ ├── config.py # 配置管理 │ ├── config.py # Configuration management
│ ├── models/ # 数据模型 │ ├── models/ # Data models
│ │ ├── __init__.py │ │ ├── __init__.py
│ │ ├── gitea.py # Gitea webhook 模型 │ │ ├── gitea.py # Gitea webhook model
│ │ └── jenkins.py # Jenkins 任务模型 │ │ └── jenkins.py # Jenkins job model
│ ├── services/ # 业务逻辑 │ ├── services/ # Business logic
│ │ ├── __init__.py │ │ ├── __init__.py
│ │ ├── webhook_service.py # Webhook 处理服务 │ │ ├── webhook_service.py # Webhook processing service
│ │ ├── jenkins_service.py # Jenkins 集成服务 │ │ ├── jenkins_service.py # Jenkins integration service
│ │ ├── queue_service.py # 队列管理服务 │ │ ├── queue_service.py # Queue management service
│ │ └── dedup_service.py # 防抖服务 │ │ └── dedup_service.py # Deduplication service
│ ├── api/ # API 路由 │ ├── api/ # API routes
│ │ ├── __init__.py │ │ ├── __init__.py
│ │ ├── webhook.py # Webhook 端点 │ │ ├── webhook.py # Webhook endpoint
│ │ ├── health.py # 健康检查 │ │ ├── health.py # Health check
│ │ └── admin.py # 管理接口 │ │ └── admin.py # Admin interface
│ ├── core/ # 核心组件 │ ├── core/ # Core components
│ │ ├── __init__.py │ │ ├── __init__.py
│ │ ├── security.py # 安全验证 │ │ ├── security.py # Security validation
│ │ ├── database.py # 数据库连接 │ │ ├── database.py # Database connection
│ │ └── cache.py # 缓存管理 │ │ └── cache.py # Cache management
│ └── tasks/ # Celery 任务 │ └── tasks/ # Celery tasks
│ ├── __init__.py │ ├── __init__.py
│ └── jenkins_tasks.py # Jenkins 任务处理 │ └── jenkins_tasks.py # Jenkins task processing
├── tests/ # 测试文件 ├── tests/ # Test files
├── docker/ # Docker 配置 ├── docker/ # Docker configuration
├── requirements.txt # Python 依赖 ├── requirements.txt # Python dependencies
├── docker-compose.yml # 开发环境 ├── docker-compose.yml # Development environment
└── README.md └── README.md
``` ```
## 🛠️ 技术栈 ## 🛠️ Tech Stack
- **Web 框架**: FastAPI - **Web Framework**: FastAPI
- **任务队列**: Celery + Redis - **Task Queue**: Celery + Redis
- **数据库**: PostgreSQL (生产) / SQLite (开发) - **Database**: PostgreSQL (production) / SQLite (development)
- **缓存**: Redis - **Cache**: Redis
- **监控**: Prometheus + Grafana - **Monitoring**: Prometheus + Grafana
- **日志**: Structured logging with JSON - **Logging**: Structured logging with JSON
- **测试**: pytest + pytest-asyncio - **Testing**: pytest + pytest-asyncio
## 🚀 快速开始 ## 🚀 Quick Start
### 1. 安装依赖 ### 1. Install Dependencies
```bash ```bash
pip install -r requirements.txt pip install -r requirements.txt
``` ```
### 2. 配置环境 ### 2. Configure Environment
```bash ```bash
cp .env.example .env cp .env.example .env
# 编辑 .env 文件配置 Jenkins 和数据库连接 # Edit the .env file to configure Jenkins and database connections
``` ```
### 3. 启动服务 ### 3. Start Service
```bash ```bash
# 启动 Redis # Start Redis
docker run -d -p 6379:6379 redis:alpine docker run -d -p 6379:6379 redis:alpine
# 启动 Celery worker # Start Celery worker
celery -A app.tasks worker --loglevel=info celery -A app.tasks worker --loglevel=info
# 启动 FastAPI 应用 # Start FastAPI application
uvicorn app.main:app --reload --host 0.0.0.0 --port 8000 uvicorn app.main:app --reload --host 0.0.0.0 --port 8000
``` ```
## 📋 配置说明 ## 📋 Configuration
### 环境分发配置 ### Environment Dispatch Configuration
```yaml ```yaml
environments: environments:
dev: dev:
@ -120,70 +120,70 @@ environments:
jenkins_url: "https://jenkins-default.example.com" jenkins_url: "https://jenkins-default.example.com"
``` ```
### 防抖配置 ### Deduplication Configuration
```yaml ```yaml
deduplication: deduplication:
enabled: true enabled: true
window_seconds: 300 # 5分钟防抖窗口 window_seconds: 300 # 5-minute deduplication window
strategy: "commit_branch" # commit_hash + branch strategy: "commit_branch" # commit_hash + branch
cache_ttl: 3600 # 缓存1小时 cache_ttl: 3600 # Cache for 1 hour
``` ```
### 队列配置 ### Queue Configuration
```yaml ```yaml
queue: queue:
max_concurrent: 10 max_concurrent: 10
max_retries: 3 max_retries: 3
retry_delay: 60 # 秒 retry_delay: 60 # seconds
priority_levels: 3 priority_levels: 3
``` ```
## 🔧 API 接口 ## 🔧 API Endpoints
### Webhook 端点 ### Webhook Endpoint
``` ```
POST /webhook/gitea POST /webhook/gitea
``` ```
### 健康检查 ### Health Check
``` ```
GET /health GET /health
GET /health/queue GET /health/queue
GET /health/jenkins GET /health/jenkins
``` ```
### 管理接口 ### Admin Endpoints
``` ```
GET /admin/queue/status GET /admin/queue/status
GET /admin/queue/stats GET /admin/queue/stats
POST /admin/queue/clear POST /admin/queue/clear
``` ```
## 🧪 测试 ## 🧪 Testing
```bash ```bash
# 运行所有测试 # Run all tests
pytest pytest
# 运行特定测试 # Run specific test
pytest tests/test_webhook_service.py pytest tests/test_webhook_service.py
# 运行性能测试 # Run performance test
pytest tests/test_performance.py pytest tests/test_performance.py
``` ```
## 📊 监控指标 ## 📊 Monitoring Metrics
- Webhook 接收率 - Webhook receive rate
- 任务队列长度 - Task queue length
- Jenkins 构建成功率 - Jenkins build success rate
- 响应时间分布 - Response time distribution
- 防抖命中率 - Deduplication hit rate
## 🔒 安全特性 ## 🔒 Security Features
- Webhook 签名验证 - Webhook signature verification
- API 密钥认证 - API key authentication
- 请求频率限制 - Request rate limiting
- 输入验证和清理 - Input validation and sanitization
- 安全日志记录 - Secure logging

292
apps/gitea-webhook-ambassador-python/README_ENHANCED.md
View File

@ -1,101 +1,101 @@
# Gitea Webhook Ambassador (Python Enhanced Version) # Gitea Webhook Ambassador (Python Enhanced Version)
这是一个用 Python 重写的 Gitea Webhook Ambassador 服务，提供与 Go 版本相同的功能，但增加了 Web 界面和更多管理功能。 This is a Gitea Webhook Ambassador service rewritten in Python, providing the same features as the Go version, but with an added web interface and more management features.
## 🚀 快速开始 ## 🚀 Quick Start
### 方式一：使用 devbox 脚本（推荐，与 Go 版本一致） ### Method 1: Use the devbox script (recommended, same as Go version)
```bash ```bash
# 安装依赖 # Install dependencies
./devbox install ./devbox install
# 初始化数据库 # Initialize database
./devbox init ./devbox init
# 启动服务 # Start the service
./devbox start ./devbox start
# 查看状态 # Check status
./devbox status ./devbox status
# 查看日志 # View logs
./devbox logs ./devbox logs
# 停止服务 # Stop the service
./devbox stop ./devbox stop
``` ```
### 方式二：使用 Makefile ### Method 2: Use Makefile
```bash ```bash
# 安装依赖 # Install dependencies
make install make install
# 初始化数据库 # Initialize database
make init make init
# 启动服务（前台运行） # Start the service (foreground)
make run make run
# 启动服务（后台运行） # Start the service (background)
make start make start
# 查看状态 # Check status
make status make status
# 查看日志 # View logs
make logs make logs
# 停止服务 # Stop the service
make stop make stop
``` ```
### 方式三：直接使用 Python ### Method 3: Use Python directly
```bash ```bash
# 创建虚拟环境 # Create virtual environment
python3 -m venv venv python3 -m venv venv
source venv/bin/activate source venv/bin/activate
# 安装依赖 # Install dependencies
pip install -r requirements.txt pip install -r requirements.txt
# 初始化数据库 # Initialize database
python -c "from app.models.database import create_tables; create_tables()" python -c "from app.models.database import create_tables; create_tables()"
# 启动服务 # Start the service
python -m uvicorn app.main_enhanced:app --host 0.0.0.0 --port 8000 python -m uvicorn app.main_enhanced:app --host 0.0.0.0 --port 8000
``` ```
## 📁 目录结构（与 Go 版本一致） ## 📁 Directory Structure (same as Go version)
``` ```
gitea-webhook-ambassador-python/ gitea-webhook-ambassador-python/
├── app/ # 应用代码 ├── app/ # Application code
│ ├── auth/ # 认证模块 │ ├── auth/ # Authentication module
│ ├── handlers/ # API 处理器 │ ├── handlers/ # API handlers
│ ├── models/ # 数据模型 │ ├── models/ # Data models
│ ├── templates/ # HTML 模板 │ ├── templates/ # HTML templates
│ ├── static/ # 静态文件 │ ├── static/ # Static files
│ └── main_enhanced.py # 主应用入口 │ └── main_enhanced.py # Main application entry
├── cmd/ # 命令行工具（与 Go 版本一致） ├── cmd/ # CLI tools (same as Go version)
│ └── server/ # 服务器启动 │ └── server/ # Server startup
├── configs/ # 配置文件（与 Go 版本一致） ├── configs/ # Configuration files (same as Go version)
│ └── config.yaml # 主配置文件 │ └── config.yaml # Main configuration file
├── data/ # 数据目录（与 Go 版本一致） ├── data/ # Data directory (same as Go version)
│ └── *.db # SQLite 数据库文件 │ └── *.db # SQLite database files
├── logs/ # 日志目录（与 Go 版本一致） ├── logs/ # Log directory (same as Go version)
│ └── service.log # 服务日志 │ └── service.log # Service log
├── devbox # 启动脚本（与 Go 版本一致） ├── devbox # Startup script (same as Go version)
├── Makefile # 构建脚本（与 Go 版本一致） ├── Makefile # Build script (same as Go version)
├── requirements.txt # Python 依赖 ├── requirements.txt # Python dependencies
└── README_ENHANCED.md # 本文档 └── README_ENHANCED.md # This document
``` ```
## 🔧 配置 ## 🔧 Configuration
编辑 `configs/config.yaml` 文件： Edit the `configs/config.yaml` file:
```yaml ```yaml
server: server:
@ -132,106 +132,106 @@ eventCleanup:
expireAfter: 7200 expireAfter: 7200
``` ```
## 🌐 Web 界面 ## 🌐 Web Interface
启动服务后，访问以下地址： After starting the service, visit the following addresses:
- **登录页面**: http://localhost:8000 - **Login page**: http://localhost:8000
- **仪表板**: http://localhost:8000/dashboard - **Dashboard**: http://localhost:8000/dashboard
- **API 文档**: http://localhost:8000/docs - **API Docs**: http://localhost:8000/docs
### 默认登录凭据 ### Default login credentials
- **用户名**: admin - **Username**: admin
- **密码**: admin-secret-key-change-in-production - **Password**: admin-secret-key-change-in-production
## 📊 功能特性 ## 📊 Features
### ✅ 与 Go 版本相同的功能 ### ✅ Same features as Go version
- Gitea Webhook 接收和处理 - Gitea Webhook receiving and processing
- Jenkins 任务触发 - Jenkins job triggering
- 项目映射配置 - Project mapping configuration
- 分支模式匹配 - Branch pattern matching
- 重试机制 - Retry mechanism
- 日志记录 - Logging
### 🆕 Python 版本增强功能 ### 🆕 Python version enhancements
- **Web 登录界面**: 基于 Bootstrap 5 的现代化界面 - **Web login interface**: Modern UI based on Bootstrap 5
- **数据库存储**: SQLite 数据库存储 API 密钥和配置 - **Database storage**: SQLite database for API keys and configuration
- **JWT 认证**: 7 天有效期的 JWT 令牌 - **JWT authentication**: 7-day valid JWT tokens
- **前端仪表板**: 多标签页管理界面 - **Frontend dashboard**: Multi-tab management interface
- **自动重定向**: 未认证用户自动跳转到登录页 - **Auto redirect**: Unauthenticated users are redirected to login
- **健康检查**: 服务状态监控 - **Health check**: Service status monitoring
- **统计信息**: 请求统计和性能指标 - **Statistics**: Request statistics and performance metrics
## 🔌 API 端点 ## 🔌 API Endpoints
### 认证相关 ### Authentication
- `POST /api/auth/login` - 用户登录 - `POST /api/auth/login` - User login
- `GET /api/auth/verify` - 验证 JWT 令牌 - `GET /api/auth/verify` - Verify JWT token
### 项目管理 ### Project Management
- `GET /api/projects` - 获取项目列表 - `GET /api/projects` - Get project list
- `POST /api/projects` - 创建新项目 - `POST /api/projects` - Create new project
- `PUT /api/projects/{id}` - 更新项目 - `PUT /api/projects/{id}` - Update project
- `DELETE /api/projects/{id}` - 删除项目 - `DELETE /api/projects/{id}` - Delete project
### API 密钥管理 ### API Key Management
- `GET /api/keys` - 获取 API 密钥列表 - `GET /api/keys` - Get API key list
- `POST /api/keys` - 创建新 API 密钥 - `POST /api/keys` - Create new API key
- `DELETE /api/keys/{id}` - 删除 API 密钥 - `DELETE /api/keys/{id}` - Delete API key
### 系统监控 ### System Monitoring
- `GET /api/health` - 健康检查 - `GET /api/health` - Health check
- `GET /api/stats` - 统计信息 - `GET /api/stats` - Statistics
- `GET /api/logs` - 日志查看 - `GET /api/logs` - View logs
### Webhook 处理 ### Webhook Handling
- `POST /webhook` - Gitea Webhook 接收端点 - `POST /webhook` - Gitea Webhook endpoint
## 🛠️ 开发 ## 🛠️ Development
### 运行测试 ### Run tests
```bash ```bash
# 使用 devbox # Use devbox
./devbox test ./devbox test
# 使用 Makefile # Use Makefile
make test make test
# 直接运行 # Run directly
python test_enhanced_features.py python test_enhanced_features.py
``` ```
### 代码检查 ### Code linting
```bash ```bash
# 使用 Makefile # Use Makefile
make lint make lint
# 直接运行 # Run directly
flake8 app/ --max-line-length=120 --ignore=E501,W503 flake8 app/ --max-line-length=120 --ignore=E501,W503
``` ```
### 清理 ### Clean up
```bash ```bash
# 使用 devbox # Use devbox
./devbox clean ./devbox clean
# 使用 Makefile # Use Makefile
make clean make clean
``` ```
## 🐳 Docker 部署 ## 🐳 Docker Deployment
### 构建镜像 ### Build image
```bash ```bash
# 使用 Makefile # Use Makefile
make docker-build make docker-build
# 直接构建 # Build directly
docker build -t gitea-webhook-ambassador:latest . docker build -t gitea-webhook-ambassador:latest .
``` ```
### 运行容器 ### Run container
```bash ```bash
docker run -d \ docker run -d \
--name gitea-webhook-ambassador \ --name gitea-webhook-ambassador \
@ -242,33 +242,33 @@ docker run -d \
gitea-webhook-ambassador:latest gitea-webhook-ambassador:latest
``` ```
## 📈 与 Go 版本对比 ## 📈 Comparison with Go Version
| 特性 | Go 版本 | Python 版本 | | Feature | Go Version | Python Version |
|------|---------|-------------| |---------|------------|---------------|
| **启动方式** | `./devbox start` | `./devbox start` | | **Startup** | `./devbox start` | `./devbox start` |
| **目录结构** | 标准 Go 项目结构 | 与 Go 版本一致 | | **Directory Structure** | Standard Go project | Same as Go version |
| **配置文件** | `configs/config.yaml` | `configs/config.yaml` | | **Config File** | `configs/config.yaml` | `configs/config.yaml` |
| **日志目录** | `logs/` | `logs/` | | **Log Directory** | `logs/` | `logs/` |
| **数据目录** | `data/` | `data/` | | **Data Directory** | `data/` | `data/` |
| **Web 界面** | ❌ 无 | ✅ 完整仪表板 | | **Web Interface** | ❌ No | ✅ Full dashboard |
| **数据库** | ❌ 无 | ✅ SQLite | | **Database** | ❌ No | ✅ SQLite |
| **JWT 认证** | ❌ 无 | ✅ 7天有效期 | | **JWT Auth** | ❌ No | ✅ 7-day validity |
| **API 密钥管理** | ❌ 无 | ✅ 数据库存储 | | **API Key Management** | ❌ No | ✅ Database storage |
| **健康检查** | ✅ 基础 | ✅ 增强版 | | **Health Check** | ✅ Basic | ✅ Enhanced |
| **性能** | 🚀 极高 | 🚀 高 | | **Performance** | 🚀 Very high | 🚀 High |
## 🔄 迁移指南 ## 🔄 Migration Guide
### 从 Go 版本迁移到 Python 版本 ### Migrate from Go version to Python version
1. **停止 Go 服务** 1. **Stop Go service**
```bash ```bash
cd /path/to/go-version cd /path/to/go-version
./devbox stop ./devbox stop
``` ```
2. **启动 Python 服务** 2. **Start Python service**
```bash ```bash
cd /path/to/python-version cd /path/to/python-version
./devbox install ./devbox install
@ -276,64 +276,64 @@ docker run -d \
./devbox start ./devbox start
``` ```
3. **验证服务** 3. **Verify service**
```bash ```bash
./devbox status ./devbox status
curl http://localhost:8000/api/health curl http://localhost:8000/api/health
``` ```
4. **配置 Webhook** 4. **Configure Webhook**
- 更新 Gitea Webhook URL 为新的 Python 服务地址 - Update Gitea Webhook URL to the new Python service address
- 确保 Jenkins 配置正确 - Ensure Jenkins configuration is correct
## 🆘 故障排除 ## 🆘 Troubleshooting
### 常见问题 ### Common Issues
**1. 端口被占用** **1. Port 8000 is occupied**
```bash ```bash
# 检查端口占用 # Check port usage
lsof -i :8000 lsof -i :8000
# 停止占用进程 # Stop the occupying process
sudo kill -9 <PID> sudo kill -9 <PID>
``` ```
**2. 虚拟环境问题** **2. Virtual environment issues**
```bash ```bash
# 重新创建虚拟环境 # Recreate virtual environment
rm -rf venv rm -rf venv
./devbox install ./devbox install
``` ```
**3. 数据库问题** **3. Database issues**
```bash ```bash
# 重新初始化数据库 # Reinitialize database
./devbox init ./devbox init
``` ```
**4. 权限问题** **4. Permission issues**
```bash ```bash
# 设置脚本权限 # Set script permissions
chmod +x devbox chmod +x devbox
``` ```
### 日志查看 ### View logs
```bash ```bash
# 查看实时日志 # View real-time logs
./devbox follow ./devbox follow
# 查看最新日志 # View latest logs
./devbox logs ./devbox logs
# 查看完整日志 # View full logs
tail -f logs/service.log tail -f logs/service.log
``` ```
## 📞 支持 ## 📞 Support
如有问题，请检查： If you have any issues, please check:
1. 服务状态：`./devbox status` 1. Service status: `./devbox status`
2. 日志信息：`./devbox logs` 2. Log information: `./devbox logs`
3. 配置文件：`configs/config.yaml` 3. Configuration file: `configs/config.yaml`
4. 网络连接：`curl http://localhost:8000/api/health` 4. Network connection: `curl http://localhost:8000/api/health`

391
apps/gitea-webhook-ambassador-python/USAGE.md
View File

@ -1,82 +1,82 @@
# 🚀 Gitea Webhook Ambassador 使用指南 # 🚀 Gitea Webhook Ambassador Usage Guide
## 📋 目录 ## 📋 Table of Contents
1. [快速开始](#快速开始) 1. [Quick Start](#quick-start)
2. [配置说明](#配置说明) 2. [Configuration](#configuration)
3. [API 接口](#api-接口) 3. [API Endpoints](#api-endpoints)
4. [数据库管理](#数据库管理) 4. [Database Management](#database-management)
5. [监控和日志](#监控和日志) 5. [Monitoring and Logs](#monitoring-and-logs)
6. [故障排除](#故障排除) 6. [Troubleshooting](#troubleshooting)
## 🚀 快速开始 ## 🚀 Quick Start
### 1. 环境准备 ### 1. Environment Preparation
```bash ```bash
# 克隆项目 # Clone the project
cd freeleaps-ops/apps/gitea-webhook-ambassador-python git clone https://your.repo.url/gitea-webhook-ambassador-python.git
cd gitea-webhook-ambassador-python
# 运行快速设置脚本 # Run quick setup script
chmod +x scripts/setup.sh ./devbox install
./scripts/setup.sh
``` ```
### 2. 配置环境 ### 2. Configure Environment
编辑 `.env` 文件，配置必要的参数： Edit the `.env` file to set required parameters:
```bash ```bash
# 编辑配置文件 # Edit configuration file
nano .env cp .env.example .env
vim .env
``` ```
**必需配置**： **Required configuration:**
```env
# Jenkins 配置 ```bash
# Jenkins configuration
JENKINS_USERNAME=your_jenkins_username JENKINS_USERNAME=your_jenkins_username
JENKINS_TOKEN=your_jenkins_api_token JENKINS_TOKEN=your_jenkins_token
# 安全配置 # Security configuration
SECURITY_SECRET_KEY=your-secret-key-here-make-it-long-and-random SECURITY_SECRET_KEY=your_secret_key
``` ```
### 3. 启动服务 ### 3. Start Service
```bash ```bash
# 方法1: 使用启动脚本 # Method 1: Use the startup script
chmod +x scripts/start.sh ./devbox start
./scripts/start.sh
# 方法2: 手动启动 # Method 2: Manual startup
# 启动 Redis # Start Redis
docker run -d --name webhook-redis -p 6379:6379 redis:alpine docker run -d -p 6379:6379 redis:alpine
# 激活虚拟环境 # Activate virtual environment
source venv/bin/activate source venv/bin/activate
# 启动 API 服务 # Start API service
python -m uvicorn app.main:app --host 0.0.0.0 --port 8000 --reload python -m uvicorn app.main_enhanced:app --host 0.0.0.0 --port 8000
# 新终端启动 Celery worker # Start Celery worker in a new terminal
celery -A app.tasks.jenkins_tasks worker --loglevel=info --concurrency=4 celery -A app.tasks.jenkins_tasks worker --loglevel=info
# 新终端启动定时任务 # Start scheduled tasks in a new terminal
celery -A app.tasks.jenkins_tasks beat --loglevel=info celery -A app.tasks.jenkins_tasks beat --loglevel=info
``` ```
### 4. 验证安装 ### 4. Verify Installation
访问以下地址验证服务是否正常： Visit the following addresses to verify the service:
- **API Docs**: http://localhost:8000/docs
- **Health Check**: http://localhost:8000/health
- **Metrics**: http://localhost:8000/metrics
- **API 文档**: http://localhost:8000/docs ## ⚙️ Configuration
- **健康检查**: http://localhost:8000/health
- **监控指标**: http://localhost:8000/metrics
## ⚙️ 配置说明 ### Environment Dispatch Configuration
### 环境分发配置 Edit the `config/environments.yaml` file:
编辑 `config/environments.yaml` 文件：
```yaml ```yaml
environments: environments:
@ -85,295 +85,174 @@ environments:
jenkins_job: "alpha-build" jenkins_job: "alpha-build"
jenkins_url: "https://jenkins-alpha.freeleaps.com" jenkins_url: "https://jenkins-alpha.freeleaps.com"
priority: 2 priority: 2
prod: prod:
branches: ["prod", "production", "main", "master", "release/*"] branches: ["prod", "production", "main", "master", "release/*"]
jenkins_job: "production-build" jenkins_job: "production-build"
jenkins_url: "https://jenkins-prod.freeleaps.com" jenkins_url: "https://jenkins-prod.freeleaps.com"
priority: 1 priority: 1
staging:
branches: ["staging", "stage", "pre-prod"]
jenkins_job: "staging-build"
jenkins_url: "https://jenkins-staging.freeleaps.com"
priority: 3
default:
branches: ["*"]
jenkins_job: "default-build"
jenkins_url: "https://jenkins-default.freeleaps.com"
priority: 4
``` ```
### 防抖配置 ### Deduplication Configuration
```yaml ```yaml
deduplication: deduplication:
enabled: true enabled: true
window_seconds: 300 # 5分钟防抖窗口 window_seconds: 300 # 5-minute deduplication window
strategy: "commit_branch" # commit_hash + branch strategy: "commit_branch"
cache_ttl: 3600 # 缓存1小时 cache_ttl: 3600 # Cache for 1 hour
``` ```
## 🔧 API 接口 ## 🔧 API Endpoints
### Webhook 端点 ### Webhook Endpoint
**POST** `/webhook/gitea` Receive Gitea webhook events:
接收 Gitea webhook 事件： ```http
POST /webhook/gitea
```bash
curl -X POST "http://localhost:8000/webhook/gitea" \
-H "Content-Type: application/json" \
-H "X-Gitea-Signature: your-secret-key" \
-d '{
"ref": "refs/heads/dev",
"before": "abc123",
"after": "def456",
"repository": {
"full_name": "freeleaps/my-project",
"clone_url": "https://gitea.freeleaps.com/freeleaps/my-project.git"
},
"pusher": {
"login": "developer",
"email": "dev@freeleaps.com"
}
}'
``` ```
### 健康检查 ### Health Check
**GET** `/health` ```http
GET /health
```bash GET /health/queue
curl http://localhost:8000/health GET /health/jenkins
``` ```
响应示例： Example response:
```json ```json
{ {
"status": "healthy", "status": "healthy",
"timestamp": 1640995200.0, "service": "Gitea Webhook Ambassador",
"services": { "version": "1.0.0",
"redis": "healthy", "timestamp": "2023-01-01T00:00:00Z",
"celery": "healthy" "jenkins": {"status": "connected"},
} "worker_pool": {"active_workers": 2, "queue_size": 0, "total_processed": 10, "total_failed": 1},
"database": {"status": "connected"}
} }
``` ```
### 队列状态 ### Queue Status
**GET** `/health/queue` ```http
GET /admin/queue/status
```bash
curl http://localhost:8000/health/queue
``` ```
响应示例： Example response:
```json ```json
{ {
"status": "healthy", "active_tasks": 1,
"queue_stats": { "queued_tasks": 2,
"active_tasks": 2, "worker_count": 2,
"queued_tasks": 5, "queue_length": 3
"worker_count": 4,
"total_queue_length": 7
}
} }
``` ```
### 监控指标 ### Monitoring Metrics
**GET** `/metrics` Returns Prometheus-formatted monitoring metrics.
```bash ## 🗄️ Database Management
curl http://localhost:8000/metrics
```
返回 Prometheus 格式的监控指标。 ### Create Project Mapping
## 🗄️ 数据库管理 Use a Python script to create a project mapping:
### 创建项目映射
使用 Python 脚本创建项目映射：
```python ```python
# create_mapping.py
import asyncio
from app.services.database_service import get_database_service from app.services.database_service import get_database_service
import asyncio
async def create_mapping():
db_service = get_database_service() db_service = get_database_service()
mapping_data = { mapping_data = {
"repository_name": "freeleaps/my-project", "repository_name": "freeleaps/test-project",
"default_job": "default-build", "default_job": "test-project-build",
"branch_jobs": [ "branch_jobs": [
{"branch_name": "dev", "job_name": "alpha-build"}, {"branch_name": "dev", "job_name": "test-project-dev"},
{"branch_name": "main", "job_name": "production-build"} {"branch_name": "staging", "job_name": "test-project-staging"}
], ],
"branch_patterns": [ "branch_patterns": [
{"pattern": r"feature/.*", "job_name": "feature-build"}, {"pattern": "feature/*", "job_name": "test-project-feature"},
{"pattern": r"hotfix/.*", "job_name": "hotfix-build"} {"pattern": "hotfix/*", "job_name": "test-project-hotfix"}
] ]
} }
success = await db_service.create_project_mapping(mapping_data) success = asyncio.run(db_service.create_project_mapping(mapping_data))
print(f"创建映射: {'成功' if success else '失败'}") print(f"Mapping created: {'Success' if success else 'Failed'}")
if __name__ == "__main__":
asyncio.run(create_mapping())
``` ```
运行脚本： Run the script:
```bash ```bash
python create_mapping.py python create_mapping.py
``` ```
### 查看触发日志 ### View Trigger Logs
```python Refer to the API documentation for log query endpoints.
# view_logs.py
import asyncio
from app.services.database_service import get_database_service
async def view_logs(): ## 📊 Monitoring and Logs
db_service = get_database_service()
logs = await db_service.get_trigger_logs( ### View Logs
repository_name="freeleaps/my-project",
limit=10
)
for log in logs:
print(f"[{log['created_at']}] {log['repository_name']} - {log['branch_name']} - {log['status']}")
if __name__ == "__main__":
asyncio.run(view_logs())
```
## 📊 监控和日志
### 日志查看
```bash ```bash
# 查看应用日志 # View application logs
tail -f logs/app.log tail -n 50 logs/service.log
# 查看 Celery 日志 # View Celery logs
tail -f logs/celery.log tail -n 50 logs/celery.log
``` ```
### 监控面板 ### Monitoring Dashboard
使用 Grafana 创建监控面板： Use Grafana to create a monitoring dashboard:
1. Visit http://localhost:3000 (Grafana)
2. Username: `admin`, Password: `admin`
3. Add Prometheus data source: http://prometheus:9090
4. Import monitoring dashboard
1. 访问 http://localhost:3000 (Grafana) ### Key Metrics
2. 用户名: `admin`, 密码: `admin` - **webhook_requests_total**: Total webhook requests
3. 添加 Prometheus 数据源: http://prometheus:9090 - **webhook_request_duration_seconds**: Request response time
4. 导入监控面板 - **queue_size**: Queue length
- **dedup_hits_total**: Deduplication hit count
### 关键指标 ## 🔧 Troubleshooting
- **webhook_requests_total**: Webhook 请求总数 ### Common Issues
- **webhook_request_duration_seconds**: 请求响应时间
- **queue_size**: 队列长度
- **dedup_hits_total**: 防抖命中次数
## 🔧 故障排除 #### 1. Redis Connection Failure
### 常见问题
#### 1. Redis 连接失败
```bash ```bash
# 检查 Redis 状态 # Check Redis status
docker ps | grep redis docker ps | grep redis
# 重启 Redis # Restart Redis
docker restart webhook-redis docker restart webhook-ambassador-redis
``` ```
#### 2. Celery Worker 无法启动 #### 2. Celery Worker Fails to Start
```bash ```bash
# 检查 Celery 配置 # Check Celery configuration
celery -A app.tasks.jenkins_tasks inspect active cat .env
# 重启 Worker # Restart Worker
pkill -f "celery.*worker"
celery -A app.tasks.jenkins_tasks worker --loglevel=info celery -A app.tasks.jenkins_tasks worker --loglevel=info
``` ```
#### 3. Jenkins 连接失败 #### 3. Jenkins Connection Failure
```bash Check Jenkins URL, username, and token in the configuration file.
# 测试 Jenkins 连接
curl -u username:token https://jenkins.example.com/api/json
```
#### 4. 数据库错误
```bash
# 检查数据库文件
ls -la webhook_ambassador.db
# 重新初始化数据库
rm webhook_ambassador.db
python -c "from app.services.database_service import get_database_service; get_database_service()"
```
### 日志级别调整
编辑 `.env` 文件：
```env
LOGGING_LEVEL=DEBUG # 开发环境
LOGGING_LEVEL=INFO # 生产环境
```
### 性能调优
#### 增加并发处理能力
```env
QUEUE_MAX_CONCURRENT=20
```
#### 调整防抖窗口
```env
DEDUPLICATION_WINDOW_SECONDS=600 # 10分钟
```
## 🐳 Docker 部署
### 使用 Docker Compose
```bash
# 启动所有服务
docker-compose up -d
# 查看服务状态
docker-compose ps
# 查看日志
docker-compose logs -f api
```
### 生产环境部署
```bash
# 构建镜像
docker build -t webhook-ambassador:latest .
# 运行容器
docker run -d \
--name webhook-ambassador \
-p 8000:8000 \
-v $(pwd)/config:/app/config \
-v $(pwd)/logs:/app/logs \
--env-file .env \
webhook-ambassador:latest
```
## 📞 支持
如果遇到问题，请检查：
1. 日志文件中的错误信息
2. 健康检查端点返回的状态
3. 监控指标中的异常数据
4. 网络连接和防火墙设置
更多帮助请参考项目文档或提交 Issue。

64
apps/gitea-webhook-ambassador-python/app/config.py
View File

@ -1,6 +1,6 @@
""" """
配置管理模块 Configuration management module
支持环境分发、防抖策略和队列配置 Supports environment dispatch, deduplication strategy, and queue configuration
""" """
from typing import Dict, List, Optional from typing import Dict, List, Optional
@ -11,7 +11,7 @@ from pathlib import Path
class EnvironmentConfig(BaseSettings): class EnvironmentConfig(BaseSettings):
"""环境配置""" """Environment configuration"""
branches: List[str] = Field(default_factory=list) branches: List[str] = Field(default_factory=list)
jenkins_job: str jenkins_job: str
jenkins_url: str jenkins_url: str
@ -19,23 +19,23 @@ class EnvironmentConfig(BaseSettings):
class DeduplicationConfig(BaseSettings): class DeduplicationConfig(BaseSettings):
"""防抖配置""" """Deduplication configuration"""
enabled: bool = True enabled: bool = True
window_seconds: int = Field(default=300, ge=1) # 5分钟防抖窗口 window_seconds: int = Field(default=300, ge=1) # 5-minute deduplication window
strategy: str = Field(default="commit_branch") # commit_hash + branch strategy: str = Field(default="commit_branch") # commit_hash + branch
cache_ttl: int = Field(default=3600, ge=1) # 缓存1小时 cache_ttl: int = Field(default=3600, ge=1) # Cache for 1 hour
class QueueConfig(BaseSettings): class QueueConfig(BaseSettings):
"""队列配置""" """Queue configuration"""
max_concurrent: int = Field(default=10, ge=1) max_concurrent: int = Field(default=10, ge=1)
max_retries: int = Field(default=3, ge=0) max_retries: int = Field(default=3, ge=0)
retry_delay: int = Field(default=60, ge=1) # 秒 retry_delay: int = Field(default=60, ge=1) # seconds
priority_levels: int = Field(default=3, ge=1, le=10) priority_levels: int = Field(default=3, ge=1, le=10)
class JenkinsConfig(BaseSettings): class JenkinsConfig(BaseSettings):
"""Jenkins 配置""" """Jenkins configuration"""
username: str username: str
token: str token: str
timeout: int = Field(default=30, ge=1) timeout: int = Field(default=30, ge=1)
@ -43,7 +43,7 @@ class JenkinsConfig(BaseSettings):
class DatabaseConfig(BaseSettings): class DatabaseConfig(BaseSettings):
"""数据库配置""" """Database configuration"""
url: str = Field(default="sqlite:///./webhook_ambassador.db") url: str = Field(default="sqlite:///./webhook_ambassador.db")
echo: bool = False echo: bool = False
pool_size: int = Field(default=10, ge=1) pool_size: int = Field(default=10, ge=1)
@ -51,74 +51,74 @@ class DatabaseConfig(BaseSettings):
class RedisConfig(BaseSettings): class RedisConfig(BaseSettings):
"""Redis 配置""" """Redis configuration"""
url: str = Field(default="redis://localhost:6379/0") url: str = Field(default="redis://localhost:6379/0")
password: Optional[str] = None password: Optional[str] = None
db: int = Field(default=0, ge=0) db: int = Field(default=0, ge=0)
class LoggingConfig(BaseSettings): class LoggingConfig(BaseSettings):
"""日志配置""" """Logging configuration"""
level: str = Field(default="INFO") level: str = Field(default="INFO")
format: str = Field(default="json") format: str = Field(default="json")
file: Optional[str] = None file: Optional[str] = None
class SecurityConfig(BaseSettings): class SecurityConfig(BaseSettings):
"""安全配置""" """Security configuration"""
secret_key: str secret_key: str
webhook_secret_header: str = Field(default="X-Gitea-Signature") webhook_secret_header: str = Field(default="X-Gitea-Signature")
rate_limit_per_minute: int = Field(default=100, ge=1) rate_limit_per_minute: int = Field(default=100, ge=1)
class Settings(BaseSettings): class Settings(BaseSettings):
"""主配置类""" """Main configuration class"""
# 基础配置 # Basic configuration
app_name: str = "Gitea Webhook Ambassador" app_name: str = "Gitea Webhook Ambassador"
version: str = "1.0.0" version: str = "1.0.0"
debug: bool = False debug: bool = False
# 服务器配置 # Server configuration
host: str = "0.0.0.0" host: str = "0.0.0.0"
port: int = Field(default=8000, ge=1, le=65535) port: int = Field(default=8000, ge=1, le=65535)
# 数据库配置 # Database configuration
database_url: str = Field(default="sqlite:///./webhook_ambassador.db") database_url: str = Field(default="sqlite:///./webhook_ambassador.db")
# Redis 配置 # Redis configuration
redis_url: str = Field(default="redis://localhost:6379/0") redis_url: str = Field(default="redis://localhost:6379/0")
redis_password: str = Field(default="") redis_password: str = Field(default="")
redis_db: int = Field(default=0) redis_db: int = Field(default=0)
# Jenkins 配置 # Jenkins configuration
jenkins_username: str = Field(default="admin") jenkins_username: str = Field(default="admin")
jenkins_token: str = Field(default="") jenkins_token: str = Field(default="")
jenkins_timeout: int = Field(default=30) jenkins_timeout: int = Field(default=30)
# 安全配置 # Security configuration
security_secret_key: str = Field(default="") security_secret_key: str = Field(default="")
security_webhook_secret_header: str = Field(default="X-Gitea-Signature") security_webhook_secret_header: str = Field(default="X-Gitea-Signature")
security_rate_limit_per_minute: int = Field(default=100) security_rate_limit_per_minute: int = Field(default=100)
# 日志配置 # Logging configuration
logging_level: str = Field(default="INFO") logging_level: str = Field(default="INFO")
logging_format: str = Field(default="json") logging_format: str = Field(default="json")
logging_file: str = Field(default="") logging_file: str = Field(default="")
# 队列配置 # Queue configuration
queue_max_concurrent: int = Field(default=10) queue_max_concurrent: int = Field(default=10)
queue_max_retries: int = Field(default=3) queue_max_retries: int = Field(default=3)
queue_retry_delay: int = Field(default=60) queue_retry_delay: int = Field(default=60)
queue_priority_levels: int = Field(default=3) queue_priority_levels: int = Field(default=3)
# 防抖配置 # Deduplication configuration
deduplication_enabled: bool = Field(default=True) deduplication_enabled: bool = Field(default=True)
deduplication_window_seconds: int = Field(default=300) deduplication_window_seconds: int = Field(default=300)
deduplication_strategy: str = Field(default="commit_branch") deduplication_strategy: str = Field(default="commit_branch")
deduplication_cache_ttl: int = Field(default=3600) deduplication_cache_ttl: int = Field(default=3600)
# 业务配置 # Business configuration
environments: Dict[str, EnvironmentConfig] = Field(default_factory=dict) environments: Dict[str, EnvironmentConfig] = Field(default_factory=dict)
deduplication: DeduplicationConfig = DeduplicationConfig() deduplication: DeduplicationConfig = DeduplicationConfig()
queue: QueueConfig = QueueConfig() queue: QueueConfig = QueueConfig()
@ -129,18 +129,18 @@ class Settings(BaseSettings):
@validator("environments", pre=True) @validator("environments", pre=True)
def load_environments_from_file(cls, v): def load_environments_from_file(cls, v):
"""从配置文件加载环境配置""" """Load environment configuration from file"""
if isinstance(v, dict) and v: if isinstance(v, dict) and v:
return v return v
# 尝试从配置文件加载 # Try to load from config file
config_file = Path("config/environments.yaml") config_file = Path("config/environments.yaml")
if config_file.exists(): if config_file.exists():
with open(config_file, "r", encoding="utf-8") as f: with open(config_file, "r", encoding="utf-8") as f:
config_data = yaml.safe_load(f) config_data = yaml.safe_load(f)
return config_data.get("environments", {}) return config_data.get("environments", {})
# 默认配置 # Default configuration
return { return {
"dev": EnvironmentConfig( "dev": EnvironmentConfig(
branches=["dev", "develop", "development"], branches=["dev", "develop", "development"],
@ -163,27 +163,27 @@ class Settings(BaseSettings):
} }
def get_environment_for_branch(self, branch: str) -> Optional[EnvironmentConfig]: def get_environment_for_branch(self, branch: str) -> Optional[EnvironmentConfig]:
"""根据分支名获取对应的环境配置""" """Get environment configuration by branch name"""
for env_name, env_config in self.environments.items(): for env_name, env_config in self.environments.items():
if branch in env_config.branches or "*" in env_config.branches: if branch in env_config.branches or "*" in env_config.branches:
return env_config return env_config
return None return None
def get_environment_by_name(self, name: str) -> Optional[EnvironmentConfig]: def get_environment_by_name(self, name: str) -> Optional[EnvironmentConfig]:
"""根据环境名获取配置""" """Get configuration by environment name"""
return self.environments.get(name) return self.environments.get(name)
# 全局配置实例 # Global configuration instance
settings = Settings() settings = Settings()
def get_settings() -> Settings: def get_settings() -> Settings:
"""获取配置实例""" """Get configuration instance"""
return settings return settings
def reload_settings(): def reload_settings():
"""重新加载配置""" """Reload configuration"""
global settings global settings
settings = Settings() settings = Settings()

4
apps/gitea-webhook-ambassador-python/app/handlers/__init__.py
View File

@ -1,6 +1,6 @@
""" """
Handlers 包 Handlers package
包含所有 API 处理器 Contains all API handlers
""" """
from . import webhook from . import webhook

52
apps/gitea-webhook-ambassador-python/app/handlers/admin.py
View File

@ -1,6 +1,6 @@
""" """
管理 API 处理器 Admin API handler
提供项目映射和 API 密钥管理功能 Provides project mapping and API key management features
""" """
import secrets import secrets
@ -17,7 +17,7 @@ from app.auth import get_current_user
router = APIRouter(prefix="/api/admin", tags=["admin"]) router = APIRouter(prefix="/api/admin", tags=["admin"])
# API 密钥相关模型 # API key related models
class APIKeyResponse(BaseModel): class APIKeyResponse(BaseModel):
id: int id: int
name: str name: str
@ -38,7 +38,7 @@ class CreateAPIKeyResponse(BaseModel):
key: str key: str
created_at: datetime created_at: datetime
# 项目映射相关模型 # Project mapping related models
class ProjectMappingRequest(BaseModel): class ProjectMappingRequest(BaseModel):
repository_name: str repository_name: str
default_job: str default_job: str
@ -57,13 +57,13 @@ class ProjectMappingResponse(BaseModel):
class Config: class Config:
from_attributes = True from_attributes = True
# API 密钥管理端点 # API key management endpoints
@router.get("/api-keys", response_model=List[APIKeyResponse]) @router.get("/api-keys", response_model=List[APIKeyResponse])
async def list_api_keys( async def list_api_keys(
db: Session = Depends(get_db), db: Session = Depends(get_db),
current_user: dict = Depends(get_current_user) current_user: dict = Depends(get_current_user)
): ):
"""列出所有 API 密钥""" """List all API keys"""
try: try:
api_keys = db.query(APIKey).order_by(APIKey.created_at.desc()).all() api_keys = db.query(APIKey).order_by(APIKey.created_at.desc()).all()
return api_keys return api_keys
@ -76,16 +76,16 @@ async def create_api_key(
db: Session = Depends(get_db), db: Session = Depends(get_db),
current_user: dict = Depends(get_current_user) current_user: dict = Depends(get_current_user)
): ):
"""创建新的 API 密钥""" """Create a new API key"""
try: try:
# 生成 API 密钥 # Generate API key
api_key = secrets.token_urlsafe(32) api_key = secrets.token_urlsafe(32)
key_prefix = api_key[:8] # 显示前8位作为前缀 key_prefix = api_key[:8] # Show first 8 characters as prefix
# 创建数据库记录 # Create database record
db_api_key = APIKey( db_api_key = APIKey(
name=request.name, name=request.name,
key_hash=api_key, # 实际应用中应该哈希存储 key_hash=api_key, # Should be hashed in production
key_prefix=key_prefix, key_prefix=key_prefix,
created_at=datetime.utcnow(), created_at=datetime.utcnow(),
last_used=datetime.utcnow(), last_used=datetime.utcnow(),
@ -99,7 +99,7 @@ async def create_api_key(
return CreateAPIKeyResponse( return CreateAPIKeyResponse(
id=db_api_key.id, id=db_api_key.id,
name=db_api_key.name, name=db_api_key.name,
key=api_key, # 只在创建时返回完整密钥 key=api_key, # Only return full key on creation
created_at=db_api_key.created_at created_at=db_api_key.created_at
) )
@ -113,7 +113,7 @@ async def delete_api_key(
db: Session = Depends(get_db), db: Session = Depends(get_db),
current_user: dict = Depends(get_current_user) current_user: dict = Depends(get_current_user)
): ):
"""删除 API 密钥""" """Delete API key"""
try: try:
api_key = db.query(APIKey).filter(APIKey.id == key_id).first() api_key = db.query(APIKey).filter(APIKey.id == key_id).first()
if not api_key: if not api_key:
@ -136,7 +136,7 @@ async def revoke_api_key(
db: Session = Depends(get_db), db: Session = Depends(get_db),
current_user: dict = Depends(get_current_user) current_user: dict = Depends(get_current_user)
): ):
"""撤销 API 密钥""" """Revoke API key"""
try: try:
api_key = db.query(APIKey).filter(APIKey.id == key_id).first() api_key = db.query(APIKey).filter(APIKey.id == key_id).first()
if not api_key: if not api_key:
@ -153,13 +153,13 @@ async def revoke_api_key(
db.rollback() db.rollback()
raise HTTPException(status_code=500, detail=f"Failed to revoke API key: {str(e)}") raise HTTPException(status_code=500, detail=f"Failed to revoke API key: {str(e)}")
# 项目映射管理端点 # Project mapping management endpoints
@router.get("/projects", response_model=List[ProjectMappingResponse]) @router.get("/projects", response_model=List[ProjectMappingResponse])
async def list_project_mappings( async def list_project_mappings(
db: Session = Depends(get_db), db: Session = Depends(get_db),
current_user: dict = Depends(get_current_user) current_user: dict = Depends(get_current_user)
): ):
"""列出所有项目映射""" """List all project mappings"""
try: try:
mappings = db.query(ProjectMapping).order_by(ProjectMapping.created_at.desc()).all() mappings = db.query(ProjectMapping).order_by(ProjectMapping.created_at.desc()).all()
return mappings return mappings
@ -172,9 +172,9 @@ async def create_project_mapping(
db: Session = Depends(get_db), db: Session = Depends(get_db),
current_user: dict = Depends(get_current_user) current_user: dict = Depends(get_current_user)
): ):
"""创建项目映射""" """Create project mapping"""
try: try:
# 检查是否已存在 # Check if already exists
existing = db.query(ProjectMapping).filter( existing = db.query(ProjectMapping).filter(
ProjectMapping.repository_name == request.repository_name ProjectMapping.repository_name == request.repository_name
).first() ).first()
@ -182,7 +182,7 @@ async def create_project_mapping(
if existing: if existing:
raise HTTPException(status_code=400, detail="Project mapping already exists") raise HTTPException(status_code=400, detail="Project mapping already exists")
# 创建新映射 # Create new mapping
mapping = ProjectMapping( mapping = ProjectMapping(
repository_name=request.repository_name, repository_name=request.repository_name,
default_job=request.default_job, default_job=request.default_job,
@ -210,7 +210,7 @@ async def get_project_mapping(
db: Session = Depends(get_db), db: Session = Depends(get_db),
current_user: dict = Depends(get_current_user) current_user: dict = Depends(get_current_user)
): ):
"""获取项目映射""" """Get project mapping"""
try: try:
mapping = db.query(ProjectMapping).filter( mapping = db.query(ProjectMapping).filter(
ProjectMapping.repository_name == repository_name ProjectMapping.repository_name == repository_name
@ -232,7 +232,7 @@ async def delete_project_mapping(
db: Session = Depends(get_db), db: Session = Depends(get_db),
current_user: dict = Depends(get_current_user) current_user: dict = Depends(get_current_user)
): ):
"""删除项目映射""" """Delete project mapping"""
try: try:
mapping = db.query(ProjectMapping).filter( mapping = db.query(ProjectMapping).filter(
ProjectMapping.repository_name == repository_name ProjectMapping.repository_name == repository_name
@ -252,24 +252,24 @@ async def delete_project_mapping(
db.rollback() db.rollback()
raise HTTPException(status_code=500, detail=f"Failed to delete project mapping: {str(e)}") raise HTTPException(status_code=500, detail=f"Failed to delete project mapping: {str(e)}")
# 统计信息端点 # Statistics endpoint
@router.get("/stats") @router.get("/stats")
async def get_admin_stats( async def get_admin_stats(
db: Session = Depends(get_db), db: Session = Depends(get_db),
current_user: dict = Depends(get_current_user) current_user: dict = Depends(get_current_user)
): ):
"""获取管理统计信息""" """Get admin statistics"""
try: try:
# API 密钥统计 # API key statistics
total_keys = db.query(APIKey).count() total_keys = db.query(APIKey).count()
active_keys = db.query(APIKey).filter(APIKey.is_active == True).count() active_keys = db.query(APIKey).filter(APIKey.is_active == True).count()
# 最近使用的密钥 # Recently used keys
recent_keys = db.query(APIKey).filter( recent_keys = db.query(APIKey).filter(
APIKey.last_used >= datetime.utcnow() - timedelta(days=7) APIKey.last_used >= datetime.utcnow() - timedelta(days=7)
).count() ).count()
# 项目映射统计 # Project mapping statistics
total_mappings = db.query(ProjectMapping).count() total_mappings = db.query(ProjectMapping).count()
return { return {

142
apps/gitea-webhook-ambassador-python/app/handlers/health.py
View File

@ -1,145 +1,21 @@
""" """
Health check handler Health check handler
Provides service health status checking Provides health check endpoints for the API
""" """
from datetime import datetime from fastapi import APIRouter
from typing import Dict, Any
from fastapi import APIRouter, Depends
from pydantic import BaseModel
from sqlalchemy.orm import Session
from app.database import get_db
from app.services.jenkins_service import get_jenkins_service
from app.services.queue_service import get_queue_service
from app.config import get_settings from app.config import get_settings
from datetime import datetime
router = APIRouter(prefix="/health", tags=["health"]) router = APIRouter()
@router.get("/health")
class JenkinsStatus(BaseModel): async def health_check():
status: str """Health check endpoint"""
message: str = None
class WorkerPoolStatus(BaseModel):
active_workers: int
queue_size: int
total_processed: int
total_failed: int
class HealthResponse(BaseModel):
status: str
service: str
version: str
timestamp: datetime
jenkins: JenkinsStatus
worker_pool: WorkerPoolStatus
database: Dict[str, Any]
@router.get("/", response_model=HealthResponse)
async def health_check(db: Session = Depends(get_db)):
"""
Health check endpoint
Check the status of each service component
"""
settings = get_settings() settings = get_settings()
# Check Jenkins connection
jenkins_service = get_jenkins_service()
jenkins_status = JenkinsStatus(status="disconnected", message="Unable to connect to Jenkins server")
try:
if await jenkins_service.test_connection():
jenkins_status = JenkinsStatus(status="connected")
except Exception as e:
jenkins_status.message = f"Connection failed: {str(e)}"
# Get worker pool stats
queue_service = get_queue_service()
try:
stats = await queue_service.get_stats()
worker_pool_status = WorkerPoolStatus(
active_workers=stats.get("active_workers", 0),
queue_size=stats.get("queue_size", 0),
total_processed=stats.get("total_processed", 0),
total_failed=stats.get("total_failed", 0)
)
except Exception as e:
worker_pool_status = WorkerPoolStatus(
active_workers=0,
queue_size=0,
total_processed=0,
total_failed=0
)
# Check database connection
database_status = {"status": "disconnected", "message": "Database connection failed"}
try:
# 尝试执行简单查询
db.execute("SELECT 1")
database_status = {"status": "connected"}
except Exception as e:
database_status["message"] = f"Database error: {str(e)}"
# Determine overall status
overall_status = "healthy"
if jenkins_status.status != "connected":
overall_status = "unhealthy"
return HealthResponse(
status=overall_status,
service="Gitea Webhook Ambassador",
version=settings.version,
timestamp=datetime.utcnow(),
jenkins=jenkins_status,
worker_pool=worker_pool_status,
database=database_status
)
@router.get("/simple")
async def simple_health_check():
"""
Simple health check endpoint
For load balancers and monitoring systems
"""
return { return {
"status": "healthy", "status": "healthy",
"service": "Gitea Webhook Ambassador", "service": "Gitea Webhook Ambassador",
"version": "1.0.0" "version": settings.version,
"timestamp": datetime.utcnow().isoformat()
} }
@router.get("/ready")
async def readiness_check(db: Session = Depends(get_db)):
"""
Readiness check endpoint
Check if the service is ready to receive requests
"""
try:
# Check database connection
db.execute("SELECT 1")
# Check Jenkins connection
jenkins_service = get_jenkins_service()
jenkins_ready = await jenkins_service.test_connection()
if jenkins_ready:
return {"status": "ready"}
else:
return {"status": "not_ready", "reason": "Jenkins connection failed"}
except Exception as e:
return {"status": "not_ready", "reason": f"Database connection failed: {str(e)}"}
@router.get("/live")
async def liveness_check():
"""
Liveness check endpoint
Check if the service process is running normally
"""
return {"status": "alive"}

16
apps/gitea-webhook-ambassador-python/app/handlers/webhook.py
View File

@ -1,6 +1,6 @@
""" """
Webhook 处理器 Webhook handler
处理来自 Gitea 的 webhook 请求 Handles webhook requests from Gitea
""" """
from fastapi import APIRouter, Depends, HTTPException, Request from fastapi import APIRouter, Depends, HTTPException, Request
@ -11,9 +11,9 @@ from app.tasks.jenkins_tasks import get_celery_app
router = APIRouter() router = APIRouter()
def get_webhook_service() -> WebhookService: def get_webhook_service() -> WebhookService:
"""获取 webhook 服务实例""" """Get webhook service instance"""
# 这里应该从依赖注入容器获取 # Should get from dependency injection container
# 暂时返回 None，实际使用时需要正确实现 # Temporarily return None, implement properly in actual use
return None return None
@router.post("/gitea") @router.post("/gitea")
@ -21,15 +21,15 @@ async def handle_gitea_webhook(
request: Request, request: Request,
webhook_service: WebhookService = Depends(get_webhook_service) webhook_service: WebhookService = Depends(get_webhook_service)
): ):
"""处理 Gitea webhook 请求""" """Handle Gitea webhook request"""
if webhook_service is None: if webhook_service is None:
raise HTTPException(status_code=503, detail="Webhook service not available") raise HTTPException(status_code=503, detail="Webhook service not available")
try: try:
# 获取请求体 # Get request body
body = await request.body() body = await request.body()
# 处理 webhook # Process webhook
result = await webhook_service.process_webhook(body, request.headers) result = await webhook_service.process_webhook(body, request.headers)
return { return {

94
apps/gitea-webhook-ambassador-python/app/main_demo.py
View File

@ -10,7 +10,7 @@ import secrets
from app.config import get_settings from app.config import get_settings
# 配置日志 # Configure logging
structlog.configure( structlog.configure(
processors=[ processors=[
structlog.stdlib.filter_by_level, structlog.stdlib.filter_by_level,
@ -31,14 +31,14 @@ structlog.configure(
logger = structlog.get_logger() logger = structlog.get_logger()
# 创建 FastAPI 应用 # Create FastAPI application
app = FastAPI( app = FastAPI(
title="Gitea Webhook Ambassador (Demo)", title="Gitea Webhook Ambassador (Demo)",
description="高性能的 Gitea 到 Jenkins 的 Webhook 服务 - 演示版本", description="High-performance webhook service from Gitea to Jenkins - Demo Version",
version="1.0.0" version="1.0.0"
) )
# 添加 CORS 中间件 # Add CORS middleware
app.add_middleware( app.add_middleware(
CORSMiddleware, CORSMiddleware,
allow_origins=["*"], allow_origins=["*"],
@ -47,13 +47,13 @@ app.add_middleware(
allow_headers=["*"], allow_headers=["*"],
) )
# 安全配置 # Security configuration
security = HTTPBearer(auto_error=False) security = HTTPBearer(auto_error=False)
# 演示数据存储 # Demo data storage
api_keys = { api_keys = {
"demo_admin_key": { "demo_admin_key": {
"name": "演示管理员密钥", "name": "Demo Admin Key",
"key_hash": "demo_admin_key", "key_hash": "demo_admin_key",
"key_prefix": "demo_adm", "key_prefix": "demo_adm",
"created_at": datetime.utcnow(), "created_at": datetime.utcnow(),
@ -62,7 +62,7 @@ api_keys = {
"role": "admin" "role": "admin"
}, },
"demo_user_key": { "demo_user_key": {
"name": "演示用户密钥", "name": "Demo User Key",
"key_hash": "demo_user_key", "key_hash": "demo_user_key",
"key_prefix": "demo_usr", "key_prefix": "demo_usr",
"created_at": datetime.utcnow(), "created_at": datetime.utcnow(),
@ -122,7 +122,7 @@ project_mappings = {
} }
} }
# 请求/响应模型 # Request/response models
class HealthResponse(BaseModel): class HealthResponse(BaseModel):
status: str status: str
service: str service: str
@ -160,12 +160,12 @@ class ProjectMappingResponse(BaseModel):
created_at: datetime created_at: datetime
updated_at: datetime updated_at: datetime
# 认证函数 # Authentication functions
def verify_api_key(api_key: str): def verify_api_key(api_key: str):
"""验证 API 密钥""" """Verify API key"""
for key_id, key_data in api_keys.items(): for key_id, key_data in api_keys.items():
if key_data["key_hash"] == api_key and key_data["is_active"]: if key_data["key_hash"] == api_key and key_data["is_active"]:
# 更新最后使用时间 # Update last used time
key_data["last_used"] = datetime.utcnow() key_data["last_used"] = datetime.utcnow()
return key_data return key_data
return None return None
@ -173,17 +173,17 @@ def verify_api_key(api_key: str):
async def get_current_user( async def get_current_user(
credentials: Optional[HTTPAuthorizationCredentials] = Depends(security) credentials: Optional[HTTPAuthorizationCredentials] = Depends(security)
): ):
"""获取当前用户（支持 API 密钥认证）""" """Get current user (supports API key authentication)"""
if not credentials: if not credentials:
raise HTTPException( raise HTTPException(
status_code=status.HTTP_401_UNAUTHORIZED, status_code=status.HTTP_401_UNAUTHORIZED,
detail="需要认证令牌", detail="Authentication token required",
headers={"WWW-Authenticate": "Bearer"}, headers={"WWW-Authenticate": "Bearer"},
) )
token = credentials.credentials token = credentials.credentials
# 验证 API 密钥 # Verify API key
api_key_data = verify_api_key(token) api_key_data = verify_api_key(token)
if api_key_data: if api_key_data:
return { return {
@ -192,17 +192,17 @@ async def get_current_user(
"role": api_key_data["role"] "role": api_key_data["role"]
} }
# 认证失败 # Authentication failed
raise HTTPException( raise HTTPException(
status_code=status.HTTP_401_UNAUTHORIZED, status_code=status.HTTP_401_UNAUTHORIZED,
detail="无效的认证令牌", detail="Invalid authentication token",
headers={"WWW-Authenticate": "Bearer"}, headers={"WWW-Authenticate": "Bearer"},
) )
# 公开端点 # Public endpoints
@app.get("/health", response_model=HealthResponse) @app.get("/health", response_model=HealthResponse)
async def health_check(): async def health_check():
"""健康检查端点""" """Health check endpoint"""
settings = get_settings() settings = get_settings()
return HealthResponse( return HealthResponse(
@ -222,11 +222,11 @@ async def health_check():
@app.get("/") @app.get("/")
async def root(): async def root():
"""根路径""" """Root path"""
return { return {
"name": "Gitea Webhook Ambassador (Demo)", "name": "Gitea Webhook Ambassador (Demo)",
"version": "1.0.0", "version": "1.0.0",
"description": "高性能的 Gitea 到 Jenkins 的 Webhook 服务 - 演示版本", "description": "High-performance webhook service from Gitea to Jenkins - Demo Version",
"endpoints": { "endpoints": {
"webhook": "/webhook/gitea", "webhook": "/webhook/gitea",
"health": "/health", "health": "/health",
@ -241,16 +241,16 @@ async def root():
@app.post("/webhook/gitea") @app.post("/webhook/gitea")
async def handle_gitea_webhook(request: Request): async def handle_gitea_webhook(request: Request):
"""处理 Gitea webhook 请求""" """Handle Gitea webhook request"""
try: try:
body = await request.body() body = await request.body()
# 记录 webhook 请求 # Log webhook request
logger.info("Received Gitea webhook", logger.info("Received Gitea webhook",
body_size=len(body), body_size=len(body),
headers=dict(request.headers)) headers=dict(request.headers))
# 添加新的触发日志 # Add new trigger log
log_entry = { log_entry = {
"id": len(trigger_logs) + 1, "id": len(trigger_logs) + 1,
"repository_name": "demo-repo", "repository_name": "demo-repo",
@ -283,7 +283,7 @@ async def handle_gitea_webhook(request: Request):
} }
) )
# 需要认证的端点 # Authenticated endpoints
@app.get("/api/logs", response_model=List[TriggerLogResponse]) @app.get("/api/logs", response_model=List[TriggerLogResponse])
async def get_trigger_logs( async def get_trigger_logs(
repository: Optional[str] = None, repository: Optional[str] = None,
@ -291,8 +291,8 @@ async def get_trigger_logs(
limit: int = 100, limit: int = 100,
current_user: dict = Depends(get_current_user) current_user: dict = Depends(get_current_user)
): ):
"""获取触发日志""" """Get trigger logs"""
print(f"用户 {current_user['username']} 访问日志端点") print(f"User {current_user['username']} accessed logs endpoint")
filtered_logs = trigger_logs.copy() filtered_logs = trigger_logs.copy()
@ -301,24 +301,24 @@ async def get_trigger_logs(
if branch: if branch:
filtered_logs = [log for log in filtered_logs if log["branch_name"] == branch] filtered_logs = [log for log in filtered_logs if log["branch_name"] == branch]
# 按时间倒序排列并限制数量 # Sort by time descending and limit
filtered_logs.sort(key=lambda x: x["created_at"], reverse=True) filtered_logs.sort(key=lambda x: x["created_at"], reverse=True)
return filtered_logs[:limit] return filtered_logs[:limit]
@app.get("/api/logs/stats") @app.get("/api/logs/stats")
async def get_log_stats(current_user: dict = Depends(get_current_user)): async def get_log_stats(current_user: dict = Depends(get_current_user)):
"""获取日志统计信息""" """Get log statistics"""
print(f"用户 {current_user['username']} 访问日志统计") print(f"User {current_user['username']} accessed log statistics")
total_logs = len(trigger_logs) total_logs = len(trigger_logs)
successful_logs = len([log for log in trigger_logs if log["status"] == "success"]) successful_logs = len([log for log in trigger_logs if log["status"] == "success"])
failed_logs = len([log for log in trigger_logs if log["status"] == "failed"]) failed_logs = len([log for log in trigger_logs if log["status"] == "failed"])
# 最近24小时的日志数 # Logs in the last 24 hours
yesterday = datetime.utcnow() - timedelta(days=1) yesterday = datetime.utcnow() - timedelta(days=1)
recent_logs = len([log for log in trigger_logs if log["created_at"] >= yesterday]) recent_logs = len([log for log in trigger_logs if log["created_at"] >= yesterday])
# 按仓库分组的统计 # Grouped by repository
repo_stats = {} repo_stats = {}
for log in trigger_logs: for log in trigger_logs:
repo = log["repository_name"] repo = log["repository_name"]
@ -337,11 +337,11 @@ async def get_log_stats(current_user: dict = Depends(get_current_user)):
@app.get("/api/admin/api-keys", response_model=List[APIKeyResponse]) @app.get("/api/admin/api-keys", response_model=List[APIKeyResponse])
async def list_api_keys(current_user: dict = Depends(get_current_user)): async def list_api_keys(current_user: dict = Depends(get_current_user)):
"""列出所有 API 密钥（仅管理员）""" """List all API keys (admin only)"""
if current_user["role"] != "admin": if current_user["role"] != "admin":
raise HTTPException(status_code=403, detail="需要管理员权限") raise HTTPException(status_code=403, detail="Admin privileges required")
print(f"管理员 {current_user['username']} 查看 API 密钥列表") print(f"Admin {current_user['username']} viewed API key list")
return [ return [
APIKeyResponse( APIKeyResponse(
@ -358,8 +358,8 @@ async def list_api_keys(current_user: dict = Depends(get_current_user)):
@app.get("/api/admin/projects", response_model=List[ProjectMappingResponse]) @app.get("/api/admin/projects", response_model=List[ProjectMappingResponse])
async def list_project_mappings(current_user: dict = Depends(get_current_user)): async def list_project_mappings(current_user: dict = Depends(get_current_user)):
"""列出所有项目映射""" """List all project mappings"""
print(f"用户 {current_user['username']} 查看项目映射") print(f"User {current_user['username']} viewed project mappings")
return [ return [
ProjectMappingResponse( ProjectMappingResponse(
@ -376,13 +376,13 @@ async def list_project_mappings(current_user: dict = Depends(get_current_user)):
@app.get("/api/admin/stats") @app.get("/api/admin/stats")
async def get_admin_stats(current_user: dict = Depends(get_current_user)): async def get_admin_stats(current_user: dict = Depends(get_current_user)):
"""获取管理统计信息""" """Get admin statistics"""
print(f"用户 {current_user['username']} 查看管理统计") print(f"User {current_user['username']} viewed admin statistics")
total_keys = len(api_keys) total_keys = len(api_keys)
active_keys = len([key for key in api_keys.values() if key["is_active"]]) active_keys = len([key for key in api_keys.values() if key["is_active"]])
# 最近使用的密钥 # Recently used keys
week_ago = datetime.utcnow() - timedelta(days=7) week_ago = datetime.utcnow() - timedelta(days=7)
recent_keys = len([ recent_keys = len([
key for key in api_keys.values() key for key in api_keys.values()
@ -402,10 +402,10 @@ async def get_admin_stats(current_user: dict = Depends(get_current_user)):
} }
} }
# 中间件 # Middleware
@app.middleware("http") @app.middleware("http")
async def log_requests(request: Request, call_next): async def log_requests(request: Request, call_next):
"""请求日志中间件""" """Request logging middleware"""
start_time = datetime.utcnow() start_time = datetime.utcnow()
response = await call_next(request) response = await call_next(request)
@ -419,13 +419,13 @@ if __name__ == "__main__":
import uvicorn import uvicorn
settings = get_settings() settings = get_settings()
print("🚀 启动 Gitea Webhook Ambassador 演示版本") print("🚀 Starting Gitea Webhook Ambassador Demo Version")
print("=" * 60) print("=" * 60)
print("📋 演示 API 密钥:") print("📋 Demo API Keys:")
print(" 管理员密钥: demo_admin_key") print(" Admin key: demo_admin_key")
print(" 用户密钥: demo_user_key") print(" User key: demo_user_key")
print() print()
print("🔧 使用示例:") print("🔧 Usage examples:")
print(" curl -H 'Authorization: Bearer demo_admin_key' http://localhost:8000/api/admin/api-keys") print(" curl -H 'Authorization: Bearer demo_admin_key' http://localhost:8000/api/admin/api-keys")
print(" curl -H 'Authorization: Bearer demo_user_key' http://localhost:8000/api/logs") print(" curl -H 'Authorization: Bearer demo_user_key' http://localhost:8000/api/logs")
print("=" * 60) print("=" * 60)

32
apps/gitea-webhook-ambassador-python/app/main_simple.py
View File

@ -1,6 +1,6 @@
""" """
简化版 FastAPI 应用主入口 Simplified FastAPI application entry point
用于快速启动和测试 For quick start and testing
""" """
from fastapi import FastAPI, Request from fastapi import FastAPI, Request
@ -15,7 +15,7 @@ from app.handlers.health import router as health_router
from app.handlers.logs import router as logs_router from app.handlers.logs import router as logs_router
from app.handlers.admin import router as admin_router from app.handlers.admin import router as admin_router
# 配置日志 # Configure logging
structlog.configure( structlog.configure(
processors=[ processors=[
structlog.stdlib.filter_by_level, structlog.stdlib.filter_by_level,
@ -36,14 +36,14 @@ structlog.configure(
logger = structlog.get_logger() logger = structlog.get_logger()
# 创建 FastAPI 应用 # Create FastAPI application
app = FastAPI( app = FastAPI(
title="Gitea Webhook Ambassador", title="Gitea Webhook Ambassador",
description="高性能的 Gitea 到 Jenkins 的 Webhook 服务", description="High-performance webhook service from Gitea to Jenkins",
version="1.0.0" version="1.0.0"
) )
# 添加 CORS 中间件 # Add CORS middleware
app.add_middleware( app.add_middleware(
CORSMiddleware, CORSMiddleware,
allow_origins=["*"], allow_origins=["*"],
@ -52,7 +52,7 @@ app.add_middleware(
allow_headers=["*"], allow_headers=["*"],
) )
# 包含路由 # Include routers
app.include_router(webhook_router) app.include_router(webhook_router)
app.include_router(health_router) app.include_router(health_router)
app.include_router(logs_router) app.include_router(logs_router)
@ -60,11 +60,11 @@ app.include_router(admin_router)
@app.get("/") @app.get("/")
async def root(): async def root():
"""根路径""" """Root path"""
return { return {
"name": "Gitea Webhook Ambassador", "name": "Gitea Webhook Ambassador",
"version": "1.0.0", "version": "1.0.0",
"description": "高性能的 Gitea 到 Jenkins 的 Webhook 服务", "description": "High-performance webhook service from Gitea to Jenkins",
"endpoints": { "endpoints": {
"webhook": "/webhook/gitea", "webhook": "/webhook/gitea",
"health": "/health", "health": "/health",
@ -76,10 +76,10 @@ async def root():
@app.middleware("http") @app.middleware("http")
async def log_requests(request: Request, call_next): async def log_requests(request: Request, call_next):
"""请求日志中间件""" """Request logging middleware"""
start_time = datetime.utcnow() start_time = datetime.utcnow()
# 记录请求 # Log request
logger.info( logger.info(
"Request started", "Request started",
method=request.method, method=request.method,
@ -87,13 +87,13 @@ async def log_requests(request: Request, call_next):
client_ip=request.client.host if request.client else None client_ip=request.client.host if request.client else None
) )
# 处理请求 # Process request
response = await call_next(request) response = await call_next(request)
# 计算处理时间 # Calculate processing time
process_time = (datetime.utcnow() - start_time).total_seconds() process_time = (datetime.utcnow() - start_time).total_seconds()
# 记录响应 # Log response
logger.info( logger.info(
"Request completed", "Request completed",
method=request.method, method=request.method,
@ -102,14 +102,14 @@ async def log_requests(request: Request, call_next):
process_time=process_time process_time=process_time
) )
# 添加处理时间到响应头 # Add processing time to response header
response.headers["X-Process-Time"] = str(process_time) response.headers["X-Process-Time"] = str(process_time)
return response return response
@app.exception_handler(Exception) @app.exception_handler(Exception)
async def global_exception_handler(request: Request, exc: Exception): async def global_exception_handler(request: Request, exc: Exception):
"""全局异常处理器""" """Global exception handler"""
logger.error( logger.error(
"Unhandled exception", "Unhandled exception",
method=request.method, method=request.method,

2
apps/gitea-webhook-ambassador-python/app/models/api_key.py
View File

@ -5,7 +5,7 @@ from sqlalchemy.ext.declarative import declarative_base
Base = declarative_base() Base = declarative_base()
class APIKey(Base): class APIKey(Base):
"""API 密钥模型""" """API key model"""
__tablename__ = "api_keys" __tablename__ = "api_keys"
id = Column(Integer, primary_key=True, index=True) id = Column(Integer, primary_key=True, index=True)

16
apps/gitea-webhook-ambassador-python/app/models/database.py
View File

@ -25,7 +25,7 @@ class ProjectMapping(Base):
created_at = Column(DateTime, default=func.now()) created_at = Column(DateTime, default=func.now())
updated_at = Column(DateTime, default=func.now(), onupdate=func.now()) updated_at = Column(DateTime, default=func.now(), onupdate=func.now())
# 关系 # Relationships
branch_jobs = relationship("BranchJob", back_populates="project", cascade="all, delete-orphan") branch_jobs = relationship("BranchJob", back_populates="project", cascade="all, delete-orphan")
branch_patterns = relationship("BranchPattern", back_populates="project", cascade="all, delete-orphan") branch_patterns = relationship("BranchPattern", back_populates="project", cascade="all, delete-orphan")
@ -39,7 +39,7 @@ class BranchJob(Base):
created_at = Column(DateTime, default=func.now()) created_at = Column(DateTime, default=func.now())
updated_at = Column(DateTime, default=func.now(), onupdate=func.now()) updated_at = Column(DateTime, default=func.now(), onupdate=func.now())
# 关系 # Relationship
project = relationship("ProjectMapping", back_populates="branch_jobs") project = relationship("ProjectMapping", back_populates="branch_jobs")
class BranchPattern(Base): class BranchPattern(Base):
@ -52,7 +52,7 @@ class BranchPattern(Base):
created_at = Column(DateTime, default=func.now()) created_at = Column(DateTime, default=func.now())
updated_at = Column(DateTime, default=func.now(), onupdate=func.now()) updated_at = Column(DateTime, default=func.now(), onupdate=func.now())
# 关系 # Relationship
project = relationship("ProjectMapping", back_populates="branch_patterns") project = relationship("ProjectMapping", back_populates="branch_patterns")
class TriggerLog(Base): class TriggerLog(Base):
@ -67,23 +67,23 @@ class TriggerLog(Base):
error_message = Column(Text, nullable=True) error_message = Column(Text, nullable=True)
created_at = Column(DateTime, default=func.now()) created_at = Column(DateTime, default=func.now())
# 数据库配置 # Database configuration
DATABASE_URL = os.getenv("DATABASE_URL", "sqlite:///./gitea_webhook_ambassador.db") DATABASE_URL = os.getenv("DATABASE_URL", "sqlite:///./gitea_webhook_ambassador.db")
# 创建引擎 # Create engine
engine = create_engine( engine = create_engine(
DATABASE_URL, DATABASE_URL,
connect_args={"check_same_thread": False} if DATABASE_URL.startswith("sqlite") else {} connect_args={"check_same_thread": False} if DATABASE_URL.startswith("sqlite") else {}
) )
# 创建会话 # Create session
SessionLocal = sessionmaker(autocommit=False, autoflush=False, bind=engine) SessionLocal = sessionmaker(autocommit=False, autoflush=False, bind=engine)
# 创建表 # Create tables
def create_tables(): def create_tables():
Base.metadata.create_all(bind=engine) Base.metadata.create_all(bind=engine)
# 获取数据库会话 # Get database session
def get_db(): def get_db():
db = SessionLocal() db = SessionLocal()
try: try:

30
apps/gitea-webhook-ambassador-python/app/models/gitea.py
View File

@ -1,5 +1,5 @@
""" """
Gitea Webhook 数据模型 Gitea Webhook data model
""" """
from typing import List, Optional from typing import List, Optional
@ -8,7 +8,7 @@ from datetime import datetime
class User(BaseModel): class User(BaseModel):
"""Gitea 用户模型""" """Gitea user model"""
id: int id: int
login: str login: str
full_name: Optional[str] = None full_name: Optional[str] = None
@ -17,7 +17,7 @@ class User(BaseModel):
class Commit(BaseModel): class Commit(BaseModel):
"""Git 提交模型""" """Git commit model"""
id: str id: str
message: str message: str
url: str url: str
@ -26,7 +26,7 @@ class Commit(BaseModel):
class Repository(BaseModel): class Repository(BaseModel):
"""Git 仓库模型""" """Git repository model"""
id: int id: int
name: str name: str
owner: User owner: User
@ -39,7 +39,7 @@ class Repository(BaseModel):
class GiteaWebhook(BaseModel): class GiteaWebhook(BaseModel):
"""Gitea Webhook 模型""" """Gitea Webhook model"""
secret: Optional[str] = None secret: Optional[str] = None
ref: str ref: str
before: str before: str
@ -50,41 +50,41 @@ class GiteaWebhook(BaseModel):
pusher: User pusher: User
def get_branch_name(self) -> str: def get_branch_name(self) -> str:
"""从 ref 中提取分支名""" """Extract branch name from ref"""
prefix = "refs/heads/" prefix = "refs/heads/"
if self.ref.startswith(prefix): if self.ref.startswith(prefix):
return self.ref[len(prefix):] return self.ref[len(prefix):]
return self.ref return self.ref
def get_event_id(self) -> str: def get_event_id(self) -> str:
"""生成唯一的事件 ID""" """Generate unique event ID"""
return f"{self.repository.full_name}-{self.after}" return f"{self.repository.full_name}-{self.after}"
def get_commit_hash(self) -> str: def get_commit_hash(self) -> str:
"""获取提交哈希""" """Get commit hash"""
return self.after return self.after
def get_deduplication_key(self) -> str: def get_deduplication_key(self) -> str:
"""生成防抖键值""" """Generate deduplication key"""
branch = self.get_branch_name() branch = self.get_branch_name()
return f"{self.after}:{branch}" return f"{self.after}:{branch}"
def is_push_event(self) -> bool: def is_push_event(self) -> bool:
"""判断是否为推送事件""" """Determine if it is a push event"""
return self.ref.startswith("refs/heads/") return self.ref.startswith("refs/heads/")
def is_tag_event(self) -> bool: def is_tag_event(self) -> bool:
"""判断是否为标签事件""" """Determine if it is a tag event"""
return self.ref.startswith("refs/tags/") return self.ref.startswith("refs/tags/")
def get_commit_message(self) -> str: def get_commit_message(self) -> str:
"""获取提交信息""" """Get commit message"""
if self.commits: if self.commits:
return self.commits[0].message return self.commits[0].message
return "" return ""
def get_author_info(self) -> dict: def get_author_info(self) -> dict:
"""获取作者信息""" """Get author information"""
if self.commits: if self.commits:
author = self.commits[0].author author = self.commits[0].author
return { return {
@ -100,7 +100,7 @@ class GiteaWebhook(BaseModel):
class WebhookEvent(BaseModel): class WebhookEvent(BaseModel):
"""Webhook 事件模型""" """Webhook event model"""
id: str id: str
repository: str repository: str
branch: str branch: str
@ -116,7 +116,7 @@ class WebhookEvent(BaseModel):
class WebhookResponse(BaseModel): class WebhookResponse(BaseModel):
"""Webhook 响应模型""" """Webhook response model"""
success: bool success: bool
message: str message: str
event_id: Optional[str] = None event_id: Optional[str] = None

2
apps/gitea-webhook-ambassador-python/app/models/project_mapping.py
View File

@ -5,7 +5,7 @@ from sqlalchemy.ext.declarative import declarative_base
Base = declarative_base() Base = declarative_base()
class ProjectMapping(Base): class ProjectMapping(Base):
"""项目映射模型""" """Project mapping model"""
__tablename__ = "project_mappings" __tablename__ = "project_mappings"
id = Column(Integer, primary_key=True, index=True) id = Column(Integer, primary_key=True, index=True)

2
apps/gitea-webhook-ambassador-python/app/models/trigger_log.py
View File

@ -5,7 +5,7 @@ from sqlalchemy.ext.declarative import declarative_base
Base = declarative_base() Base = declarative_base()
class TriggerLog(Base): class TriggerLog(Base):
"""触发日志模型""" """Trigger log model"""
__tablename__ = "trigger_logs" __tablename__ = "trigger_logs"
id = Column(Integer, primary_key=True, index=True) id = Column(Integer, primary_key=True, index=True)

112
apps/gitea-webhook-ambassador-python/app/services/database_service.py
View File

@ -1,6 +1,6 @@
""" """
数据库服务 Database service
实现项目映射、分支模式匹配等功能 Implements project mapping, branch pattern matching, and related features
""" """
import asyncio import asyncio
@ -18,10 +18,9 @@ from app.config import get_settings
logger = structlog.get_logger() logger = structlog.get_logger()
Base = declarative_base() Base = declarative_base()
# Database models
# 数据库模型
class APIKey(Base): class APIKey(Base):
"""API 密钥模型""" """API key model"""
__tablename__ = "api_keys" __tablename__ = "api_keys"
id = Column(Integer, primary_key=True, autoincrement=True) id = Column(Integer, primary_key=True, autoincrement=True)
@ -30,9 +29,8 @@ class APIKey(Base):
created_at = Column(DateTime, default=datetime.utcnow) created_at = Column(DateTime, default=datetime.utcnow)
updated_at = Column(DateTime, default=datetime.utcnow, onupdate=datetime.utcnow) updated_at = Column(DateTime, default=datetime.utcnow, onupdate=datetime.utcnow)
class ProjectMapping(Base): class ProjectMapping(Base):
"""项目映射模型""" """Project mapping model"""
__tablename__ = "project_mappings" __tablename__ = "project_mappings"
id = Column(Integer, primary_key=True, autoincrement=True) id = Column(Integer, primary_key=True, autoincrement=True)
@ -41,13 +39,12 @@ class ProjectMapping(Base):
created_at = Column(DateTime, default=datetime.utcnow) created_at = Column(DateTime, default=datetime.utcnow)
updated_at = Column(DateTime, default=datetime.utcnow, onupdate=datetime.utcnow) updated_at = Column(DateTime, default=datetime.utcnow, onupdate=datetime.utcnow)
# 关系 # Relationships
branch_jobs = relationship("BranchJob", back_populates="project", cascade="all, delete-orphan") branch_jobs = relationship("BranchJob", back_populates="project", cascade="all, delete-orphan")
branch_patterns = relationship("BranchPattern", back_populates="project", cascade="all, delete-orphan") branch_patterns = relationship("BranchPattern", back_populates="project", cascade="all, delete-orphan")
class BranchJob(Base): class BranchJob(Base):
"""分支任务映射模型""" """Branch job mapping model"""
__tablename__ = "branch_jobs" __tablename__ = "branch_jobs"
id = Column(Integer, primary_key=True, autoincrement=True) id = Column(Integer, primary_key=True, autoincrement=True)
@ -57,12 +54,11 @@ class BranchJob(Base):
created_at = Column(DateTime, default=datetime.utcnow) created_at = Column(DateTime, default=datetime.utcnow)
updated_at = Column(DateTime, default=datetime.utcnow, onupdate=datetime.utcnow) updated_at = Column(DateTime, default=datetime.utcnow, onupdate=datetime.utcnow)
# 关系 # Relationship
project = relationship("ProjectMapping", back_populates="branch_jobs") project = relationship("ProjectMapping", back_populates="branch_jobs")
class BranchPattern(Base): class BranchPattern(Base):
"""分支模式映射模型""" """Branch pattern mapping model"""
__tablename__ = "branch_patterns" __tablename__ = "branch_patterns"
id = Column(Integer, primary_key=True, autoincrement=True) id = Column(Integer, primary_key=True, autoincrement=True)
@ -72,12 +68,11 @@ class BranchPattern(Base):
created_at = Column(DateTime, default=datetime.utcnow) created_at = Column(DateTime, default=datetime.utcnow)
updated_at = Column(DateTime, default=datetime.utcnow, onupdate=datetime.utcnow) updated_at = Column(DateTime, default=datetime.utcnow, onupdate=datetime.utcnow)
# 关系 # Relationship
project = relationship("ProjectMapping", back_populates="branch_patterns") project = relationship("ProjectMapping", back_populates="branch_patterns")
class TriggerLog(Base): class TriggerLog(Base):
"""触发日志模型""" """Trigger log model"""
__tablename__ = "trigger_logs" __tablename__ = "trigger_logs"
id = Column(Integer, primary_key=True, autoincrement=True) id = Column(Integer, primary_key=True, autoincrement=True)
@ -89,9 +84,8 @@ class TriggerLog(Base):
error_message = Column(Text) error_message = Column(Text)
created_at = Column(DateTime, default=datetime.utcnow) created_at = Column(DateTime, default=datetime.utcnow)
class DatabaseService: class DatabaseService:
"""数据库服务""" """Database service"""
def __init__(self): def __init__(self):
self.settings = get_settings() self.settings = get_settings()
@ -100,7 +94,7 @@ class DatabaseService:
self._init_database() self._init_database()
def _init_database(self): def _init_database(self):
"""初始化数据库""" """Initialize database"""
try: try:
self.engine = create_engine( self.engine = create_engine(
self.settings.database.url, self.settings.database.url,
@ -109,10 +103,10 @@ class DatabaseService:
max_overflow=self.settings.database.max_overflow max_overflow=self.settings.database.max_overflow
) )
# 创建表 # Create tables
Base.metadata.create_all(bind=self.engine) Base.metadata.create_all(bind=self.engine)
# 创建会话工厂 # Create session factory
self.SessionLocal = sessionmaker(autocommit=False, autoflush=False, bind=self.engine) self.SessionLocal = sessionmaker(autocommit=False, autoflush=False, bind=self.engine)
logger.info("Database initialized successfully") logger.info("Database initialized successfully")
@ -122,18 +116,16 @@ class DatabaseService:
raise raise
def get_session(self): def get_session(self):
"""获取数据库会话""" """Get database session"""
return self.SessionLocal() return self.SessionLocal()
async def get_project_mapping(self, repository_name: str) -> Optional[Dict[str, Any]]: async def get_project_mapping(self, repository_name: str) -> Optional[Dict[str, Any]]:
""" """
获取项目映射 Get project mapping
Args: Args:
repository_name: 仓库名 repository_name: repository name
Returns: Returns:
Dict: 项目映射信息 Dict: project mapping info
""" """
try: try:
def _get_mapping(): def _get_mapping():
@ -146,7 +138,7 @@ class DatabaseService:
if not project: if not project:
return None return None
# 构建返回数据 # Build return data
result = { result = {
"id": project.id, "id": project.id,
"repository_name": project.repository_name, "repository_name": project.repository_name,
@ -155,7 +147,7 @@ class DatabaseService:
"branch_patterns": [] "branch_patterns": []
} }
# 添加分支任务映射 # Add branch job mappings
for branch_job in project.branch_jobs: for branch_job in project.branch_jobs:
result["branch_jobs"].append({ result["branch_jobs"].append({
"id": branch_job.id, "id": branch_job.id,
@ -163,7 +155,7 @@ class DatabaseService:
"job_name": branch_job.job_name "job_name": branch_job.job_name
}) })
# 添加分支模式映射 # Add branch pattern mappings
for pattern in project.branch_patterns: for pattern in project.branch_patterns:
result["branch_patterns"].append({ result["branch_patterns"].append({
"id": pattern.id, "id": pattern.id,
@ -176,7 +168,7 @@ class DatabaseService:
finally: finally:
session.close() session.close()
# 在线程池中执行数据库操作 # Run DB operation in thread pool
loop = asyncio.get_event_loop() loop = asyncio.get_event_loop()
return await loop.run_in_executor(None, _get_mapping) return await loop.run_in_executor(None, _get_mapping)
@ -187,28 +179,26 @@ class DatabaseService:
async def determine_job_name(self, repository_name: str, branch_name: str) -> Optional[str]: async def determine_job_name(self, repository_name: str, branch_name: str) -> Optional[str]:
""" """
根据分支名确定任务名 Determine job name by branch
Args: Args:
repository_name: 仓库名 repository_name: repository name
branch_name: 分支名 branch_name: branch name
Returns: Returns:
str: 任务名 str: job name
""" """
try: try:
project = await self.get_project_mapping(repository_name) project = await self.get_project_mapping(repository_name)
if not project: if not project:
return None return None
# 1. 检查精确分支匹配 # 1. Check exact branch match
for branch_job in project["branch_jobs"]: for branch_job in project["branch_jobs"]:
if branch_job["branch_name"] == branch_name: if branch_job["branch_name"] == branch_name:
logger.debug("Found exact branch match", logger.debug("Found exact branch match",
branch=branch_name, job=branch_job["job_name"]) branch=branch_name, job=branch_job["job_name"])
return branch_job["job_name"] return branch_job["job_name"]
# 2. 检查模式匹配 # 2. Check pattern match
for pattern in project["branch_patterns"]: for pattern in project["branch_patterns"]:
try: try:
if re.match(pattern["pattern"], branch_name): if re.match(pattern["pattern"], branch_name):
@ -221,7 +211,7 @@ class DatabaseService:
pattern=pattern["pattern"], error=str(e)) pattern=pattern["pattern"], error=str(e))
continue continue
# 3. 使用默认任务 # 3. Use default job
if project["default_job"]: if project["default_job"]:
logger.debug("Using default job", logger.debug("Using default job",
branch=branch_name, job=project["default_job"]) branch=branch_name, job=project["default_job"])
@ -237,13 +227,11 @@ class DatabaseService:
async def log_trigger(self, log_data: Dict[str, Any]) -> bool: async def log_trigger(self, log_data: Dict[str, Any]) -> bool:
""" """
记录触发日志 Log trigger event
Args: Args:
log_data: 日志数据 log_data: log data
Returns: Returns:
bool: 是否成功 bool: success or not
""" """
try: try:
def _log_trigger(): def _log_trigger():
@ -279,15 +267,13 @@ class DatabaseService:
async def get_trigger_logs(self, repository_name: str = None, async def get_trigger_logs(self, repository_name: str = None,
branch_name: str = None, limit: int = 100) -> List[Dict[str, Any]]: branch_name: str = None, limit: int = 100) -> List[Dict[str, Any]]:
""" """
获取触发日志 Get trigger logs
Args: Args:
repository_name: 仓库名（可选） repository_name: repository name (optional)
branch_name: 分支名（可选） branch_name: branch name (optional)
limit: 限制数量 limit: limit number
Returns: Returns:
List: 日志列表 List: log list
""" """
try: try:
def _get_logs(): def _get_logs():
@ -329,28 +315,26 @@ class DatabaseService:
async def create_project_mapping(self, mapping_data: Dict[str, Any]) -> bool: async def create_project_mapping(self, mapping_data: Dict[str, Any]) -> bool:
""" """
创建项目映射 Create project mapping
Args: Args:
mapping_data: 映射数据 mapping_data: mapping data
Returns: Returns:
bool: 是否成功 bool: success or not
""" """
try: try:
def _create_mapping(): def _create_mapping():
session = self.get_session() session = self.get_session()
try: try:
# 创建项目映射 # Create project mapping
project = ProjectMapping( project = ProjectMapping(
repository_name=mapping_data["repository_name"], repository_name=mapping_data["repository_name"],
default_job=mapping_data.get("default_job") default_job=mapping_data.get("default_job")
) )
session.add(project) session.add(project)
session.flush() # 获取 ID session.flush() # Get ID
# 添加分支任务映射 # Add branch job mappings
for branch_job in mapping_data.get("branch_jobs", []): for branch_job in mapping_data.get("branch_jobs", []):
job = BranchJob( job = BranchJob(
project_id=project.id, project_id=project.id,
@ -359,7 +343,7 @@ class DatabaseService:
) )
session.add(job) session.add(job)
# 添加分支模式映射 # Add branch pattern mappings
for pattern in mapping_data.get("branch_patterns", []): for pattern in mapping_data.get("branch_patterns", []):
pattern_obj = BranchPattern( pattern_obj = BranchPattern(
project_id=project.id, project_id=project.id,
@ -385,13 +369,11 @@ class DatabaseService:
logger.error("Failed to create project mapping", error=str(e)) logger.error("Failed to create project mapping", error=str(e))
return False return False
# Global database service instance
# 全局数据库服务实例
_database_service: Optional[DatabaseService] = None _database_service: Optional[DatabaseService] = None
def get_database_service() -> DatabaseService: def get_database_service() -> DatabaseService:
"""获取数据库服务实例""" """Get database service instance"""
global _database_service global _database_service
if _database_service is None: if _database_service is None:
_database_service = DatabaseService() _database_service = DatabaseService()

78
apps/gitea-webhook-ambassador-python/app/services/dedup_service.py
View File

@ -1,6 +1,6 @@
""" """
防抖服务 Deduplication service
实现基于 commit hash + 分支的去重策略 Implements deduplication strategy based on commit hash + branch
""" """
import asyncio import asyncio
@ -15,9 +15,8 @@ from app.config import get_settings
logger = structlog.get_logger() logger = structlog.get_logger()
class DeduplicationService: class DeduplicationService:
"""防抖服务""" """Deduplication service"""
def __init__(self, redis_client: aioredis.Redis): def __init__(self, redis_client: aioredis.Redis):
self.redis = redis_client self.redis = redis_client
@ -26,13 +25,11 @@ class DeduplicationService:
async def is_duplicate(self, dedup_key: str) -> bool: async def is_duplicate(self, dedup_key: str) -> bool:
""" """
检查是否为重复事件 Check if the event is a duplicate
Args: Args:
dedup_key: 防抖键值 (commit_hash:branch) dedup_key: deduplication key (commit_hash:branch)
Returns: Returns:
bool: True 表示重复，False 表示新事件 bool: True if duplicate, False if new event
""" """
if not self.settings.deduplication.enabled: if not self.settings.deduplication.enabled:
return False return False
@ -40,13 +37,13 @@ class DeduplicationService:
try: try:
cache_key = f"{self.cache_prefix}{dedup_key}" cache_key = f"{self.cache_prefix}{dedup_key}"
# 检查是否在缓存中 # Check if in cache
exists = await self.redis.exists(cache_key) exists = await self.redis.exists(cache_key)
if exists: if exists:
logger.info("Duplicate event detected", dedup_key=dedup_key) logger.info("Duplicate event detected", dedup_key=dedup_key)
return True return True
# 记录新事件 # Record new event
await self._record_event(cache_key, dedup_key) await self._record_event(cache_key, dedup_key)
logger.info("New event recorded", dedup_key=dedup_key) logger.info("New event recorded", dedup_key=dedup_key)
return False return False
@ -54,13 +51,13 @@ class DeduplicationService:
except Exception as e: except Exception as e:
logger.error("Error checking duplication", logger.error("Error checking duplication",
dedup_key=dedup_key, error=str(e)) dedup_key=dedup_key, error=str(e))
# 出错时允许通过，避免阻塞 # Allow through on error to avoid blocking
return False return False
async def _record_event(self, cache_key: str, dedup_key: str): async def _record_event(self, cache_key: str, dedup_key: str):
"""记录事件到缓存""" """Record event to cache"""
try: try:
# 设置缓存，TTL 为防抖窗口时间 # Set cache, TTL is deduplication window
ttl = self.settings.deduplication.cache_ttl ttl = self.settings.deduplication.cache_ttl
await self.redis.setex(cache_key, ttl, json.dumps({ await self.redis.setex(cache_key, ttl, json.dumps({
"dedup_key": dedup_key, "dedup_key": dedup_key,
@ -68,7 +65,7 @@ class DeduplicationService:
"ttl": ttl "ttl": ttl
})) }))
# 同时记录到时间窗口缓存 # Also record to window cache
window_key = f"{self.cache_prefix}window:{dedup_key}" window_key = f"{self.cache_prefix}window:{dedup_key}"
window_ttl = self.settings.deduplication.window_seconds window_ttl = self.settings.deduplication.window_seconds
await self.redis.setex(window_key, window_ttl, "1") await self.redis.setex(window_key, window_ttl, "1")
@ -78,7 +75,7 @@ class DeduplicationService:
cache_key=cache_key, error=str(e)) cache_key=cache_key, error=str(e))
async def get_event_info(self, dedup_key: str) -> Optional[Dict[str, Any]]: async def get_event_info(self, dedup_key: str) -> Optional[Dict[str, Any]]:
"""获取事件信息""" """Get event info"""
try: try:
cache_key = f"{self.cache_prefix}{dedup_key}" cache_key = f"{self.cache_prefix}{dedup_key}"
data = await self.redis.get(cache_key) data = await self.redis.get(cache_key)
@ -91,12 +88,12 @@ class DeduplicationService:
return None return None
async def clear_event(self, dedup_key: str) -> bool: async def clear_event(self, dedup_key: str) -> bool:
"""清除事件记录""" """Clear event record"""
try: try:
cache_key = f"{self.cache_prefix}{dedup_key}" cache_key = f"{self.cache_prefix}{dedup_key}"
window_key = f"{self.cache_prefix}window:{dedup_key}" window_key = f"{self.cache_prefix}window:{dedup_key}"
# 删除两个缓存键 # Delete both cache keys
await self.redis.delete(cache_key, window_key) await self.redis.delete(cache_key, window_key)
logger.info("Event cleared", dedup_key=dedup_key) logger.info("Event cleared", dedup_key=dedup_key)
return True return True
@ -107,18 +104,18 @@ class DeduplicationService:
return False return False
async def get_stats(self) -> Dict[str, Any]: async def get_stats(self) -> Dict[str, Any]:
"""获取防抖统计信息""" """Get deduplication statistics"""
try: try:
# 获取所有防抖键 # Get all deduplication keys
pattern = f"{self.cache_prefix}*" pattern = f"{self.cache_prefix}*"
keys = await self.redis.keys(pattern) keys = await self.redis.keys(pattern)
# 统计不同类型的键 # Count different types of keys
total_keys = len(keys) total_keys = len(keys)
window_keys = len([k for k in keys if b"window:" in k]) window_keys = len([k for k in keys if b"window:" in k])
event_keys = total_keys - window_keys event_keys = total_keys - window_keys
# 获取配置信息 # Get config info
config = { config = {
"enabled": self.settings.deduplication.enabled, "enabled": self.settings.deduplication.enabled,
"window_seconds": self.settings.deduplication.window_seconds, "window_seconds": self.settings.deduplication.window_seconds,
@ -139,14 +136,14 @@ class DeduplicationService:
return {"error": str(e)} return {"error": str(e)}
async def cleanup_expired_events(self) -> int: async def cleanup_expired_events(self) -> int:
"""清理过期事件""" """Clean up expired events"""
try: try:
pattern = f"{self.cache_prefix}*" pattern = f"{self.cache_prefix}*"
keys = await self.redis.keys(pattern) keys = await self.redis.keys(pattern)
cleaned_count = 0 cleaned_count = 0
for key in keys: for key in keys:
# 检查 TTL # Check TTL
ttl = await self.redis.ttl(key) ttl = await self.redis.ttl(key)
if ttl <= 0: if ttl <= 0:
await self.redis.delete(key) await self.redis.delete(key)
@ -163,14 +160,12 @@ class DeduplicationService:
def generate_dedup_key(self, commit_hash: str, branch: str) -> str: def generate_dedup_key(self, commit_hash: str, branch: str) -> str:
""" """
生成防抖键值 Generate deduplication key
Args: Args:
commit_hash: 提交哈希 commit_hash: commit hash
branch: 分支名 branch: branch name
Returns: Returns:
str: 防抖键值 str: deduplication key
""" """
if self.settings.deduplication.strategy == "commit_branch": if self.settings.deduplication.strategy == "commit_branch":
return f"{commit_hash}:{branch}" return f"{commit_hash}:{branch}"
@ -179,18 +174,16 @@ class DeduplicationService:
elif self.settings.deduplication.strategy == "branch_only": elif self.settings.deduplication.strategy == "branch_only":
return branch return branch
else: else:
# 默认使用 commit_hash:branch # Default use commit_hash:branch
return f"{commit_hash}:{branch}" return f"{commit_hash}:{branch}"
async def is_in_window(self, dedup_key: str) -> bool: async def is_in_window(self, dedup_key: str) -> bool:
""" """
检查是否在防抖时间窗口内 Check if in deduplication time window
Args: Args:
dedup_key: 防抖键值 dedup_key: deduplication key
Returns: Returns:
bool: True 表示在窗口内 bool: True if in window
""" """
try: try:
window_key = f"{self.cache_prefix}window:{dedup_key}" window_key = f"{self.cache_prefix}window:{dedup_key}"
@ -202,22 +195,19 @@ class DeduplicationService:
dedup_key=dedup_key, error=str(e)) dedup_key=dedup_key, error=str(e))
return False return False
# Global deduplication service instance
# 全局防抖服务实例
_dedup_service: Optional[DeduplicationService] = None _dedup_service: Optional[DeduplicationService] = None
def get_deduplication_service() -> DeduplicationService: def get_deduplication_service() -> DeduplicationService:
"""获取防抖服务实例""" """Get deduplication service instance"""
global _dedup_service global _dedup_service
if _dedup_service is None: if _dedup_service is None:
# 这里需要从依赖注入获取 Redis 客户端 # Should get Redis client from dependency injection
# 在实际使用时，应该通过依赖注入传入 # In actual use, should be passed in
raise RuntimeError("DeduplicationService not initialized") raise RuntimeError("DeduplicationService not initialized")
return _dedup_service return _dedup_service
def set_deduplication_service(service: DeduplicationService): def set_deduplication_service(service: DeduplicationService):
"""设置防抖服务实例""" """Set deduplication service instance"""
global _dedup_service global _dedup_service
_dedup_service = service _dedup_service = service

20
apps/gitea-webhook-ambassador-python/app/services/jenkins_service.py
View File

@ -1,6 +1,6 @@
""" """
Jenkins 服务 Jenkins service
提供与 Jenkins 的交互功能 Provides interaction with Jenkins
""" """
import aiohttp import aiohttp
@ -12,7 +12,7 @@ from app.config import get_settings
logger = structlog.get_logger() logger = structlog.get_logger()
class JenkinsService: class JenkinsService:
"""Jenkins 服务类""" """Jenkins service class"""
def __init__(self): def __init__(self):
self.settings = get_settings() self.settings = get_settings()
@ -22,7 +22,7 @@ class JenkinsService:
self.timeout = self.settings.jenkins.timeout self.timeout = self.settings.jenkins.timeout
async def test_connection(self) -> bool: async def test_connection(self) -> bool:
"""测试 Jenkins 连接""" """Test Jenkins connection"""
try: try:
async with aiohttp.ClientSession() as session: async with aiohttp.ClientSession() as session:
auth = aiohttp.BasicAuth(self.username, self.token) auth = aiohttp.BasicAuth(self.username, self.token)
@ -42,15 +42,15 @@ class JenkinsService:
return False return False
async def trigger_job(self, job_name: str, parameters: Optional[Dict[str, Any]] = None) -> bool: async def trigger_job(self, job_name: str, parameters: Optional[Dict[str, Any]] = None) -> bool:
"""触发 Jenkins 任务""" """Trigger Jenkins job"""
try: try:
async with aiohttp.ClientSession() as session: async with aiohttp.ClientSession() as session:
auth = aiohttp.BasicAuth(self.username, self.token) auth = aiohttp.BasicAuth(self.username, self.token)
# 构建请求 URL # Build request URL
url = f"{self.base_url}/job/{job_name}/build" url = f"{self.base_url}/job/{job_name}/build"
# 如果有参数，使用参数化构建 # If parameters, use parameterized build
if parameters: if parameters:
url = f"{self.base_url}/job/{job_name}/buildWithParameters" url = f"{self.base_url}/job/{job_name}/buildWithParameters"
@ -71,7 +71,7 @@ class JenkinsService:
return False return False
async def get_job_info(self, job_name: str) -> Optional[Dict[str, Any]]: async def get_job_info(self, job_name: str) -> Optional[Dict[str, Any]]:
"""获取任务信息""" """Get job info"""
try: try:
async with aiohttp.ClientSession() as session: async with aiohttp.ClientSession() as session:
auth = aiohttp.BasicAuth(self.username, self.token) auth = aiohttp.BasicAuth(self.username, self.token)
@ -89,11 +89,11 @@ class JenkinsService:
logger.error(f"Error getting job info for {job_name}: {str(e)}") logger.error(f"Error getting job info for {job_name}: {str(e)}")
return None return None
# 全局服务实例 # Global service instance
_jenkins_service = None _jenkins_service = None
def get_jenkins_service() -> JenkinsService: def get_jenkins_service() -> JenkinsService:
"""获取 Jenkins 服务实例""" """Get Jenkins service instance"""
global _jenkins_service global _jenkins_service
if _jenkins_service is None: if _jenkins_service is None:
_jenkins_service = JenkinsService() _jenkins_service = JenkinsService()

24
apps/gitea-webhook-ambassador-python/app/services/queue_service.py
View File

@ -1,6 +1,6 @@
""" """
队列服务 Queue service
提供任务队列管理功能 Provides task queue management features
""" """
import structlog import structlog
@ -10,7 +10,7 @@ from datetime import datetime
logger = structlog.get_logger() logger = structlog.get_logger()
class QueueService: class QueueService:
"""队列服务类""" """Queue service class"""
def __init__(self): def __init__(self):
self.active_workers = 0 self.active_workers = 0
@ -25,45 +25,45 @@ class QueueService:
} }
async def get_stats(self) -> Dict[str, Any]: async def get_stats(self) -> Dict[str, Any]:
"""获取队列统计信息""" """Get queue statistics"""
return self._stats.copy() return self._stats.copy()
async def increment_processed(self): async def increment_processed(self):
"""增加已处理任务计数""" """Increase processed task count"""
self.total_processed += 1 self.total_processed += 1
self._stats["total_processed"] = self.total_processed self._stats["total_processed"] = self.total_processed
async def increment_failed(self): async def increment_failed(self):
"""增加失败任务计数""" """Increase failed task count"""
self.total_failed += 1 self.total_failed += 1
self._stats["total_failed"] = self.total_failed self._stats["total_failed"] = self.total_failed
async def set_active_workers(self, count: int): async def set_active_workers(self, count: int):
"""设置活跃工作线程数""" """Set number of active workers"""
self.active_workers = count self.active_workers = count
self._stats["active_workers"] = count self._stats["active_workers"] = count
async def set_queue_size(self, size: int): async def set_queue_size(self, size: int):
"""设置队列大小""" """Set queue size"""
self.queue_size = size self.queue_size = size
self._stats["queue_size"] = size self._stats["queue_size"] = size
async def add_to_queue(self): async def add_to_queue(self):
"""添加任务到队列""" """Add task to queue"""
self.queue_size += 1 self.queue_size += 1
self._stats["queue_size"] = self.queue_size self._stats["queue_size"] = self.queue_size
async def remove_from_queue(self): async def remove_from_queue(self):
"""从队列移除任务""" """Remove task from queue"""
if self.queue_size > 0: if self.queue_size > 0:
self.queue_size -= 1 self.queue_size -= 1
self._stats["queue_size"] = self.queue_size self._stats["queue_size"] = self.queue_size
# 全局服务实例 # Global service instance
_queue_service = None _queue_service = None
def get_queue_service() -> QueueService: def get_queue_service() -> QueueService:
"""获取队列服务实例""" """Get queue service instance"""
global _queue_service global _queue_service
if _queue_service is None: if _queue_service is None:
_queue_service = QueueService() _queue_service = QueueService()

67
apps/gitea-webhook-ambassador-python/app/services/webhook_service.py
View File

@ -1,6 +1,6 @@
""" """
Webhook 处理服务 Webhook processing service
实现智能分发、任务排队和防抖策略 Implements intelligent dispatch, task queueing, and deduplication strategy
""" """
import asyncio import asyncio
@ -18,9 +18,8 @@ from app.tasks.jenkins_tasks import trigger_jenkins_job
logger = structlog.get_logger() logger = structlog.get_logger()
class WebhookService: class WebhookService:
"""Webhook 处理服务""" """Webhook processing service"""
def __init__( def __init__(
self, self,
@ -36,16 +35,14 @@ class WebhookService:
async def process_webhook(self, webhook: GiteaWebhook) -> WebhookResponse: async def process_webhook(self, webhook: GiteaWebhook) -> WebhookResponse:
""" """
处理 Webhook 事件 Process webhook event
Args: Args:
webhook: Gitea Webhook 数据 webhook: Gitea webhook data
Returns: Returns:
WebhookResponse: 处理结果 WebhookResponse: processing result
""" """
try: try:
# 1. 验证事件类型 # 1. Validate event type
if not webhook.is_push_event(): if not webhook.is_push_event():
return WebhookResponse( return WebhookResponse(
success=True, success=True,
@ -53,7 +50,7 @@ class WebhookService:
event_id=webhook.get_event_id() event_id=webhook.get_event_id()
) )
# 2. 提取关键信息 # 2. Extract key information
branch = webhook.get_branch_name() branch = webhook.get_branch_name()
commit_hash = webhook.get_commit_hash() commit_hash = webhook.get_commit_hash()
repository = webhook.repository.full_name repository = webhook.repository.full_name
@ -63,7 +60,7 @@ class WebhookService:
branch=branch, branch=branch,
commit_hash=commit_hash) commit_hash=commit_hash)
# 3. 防抖检查 # 3. Deduplication check
dedup_key = self.dedup_service.generate_dedup_key(commit_hash, branch) dedup_key = self.dedup_service.generate_dedup_key(commit_hash, branch)
if await self.dedup_service.is_duplicate(dedup_key): if await self.dedup_service.is_duplicate(dedup_key):
return WebhookResponse( return WebhookResponse(
@ -72,7 +69,7 @@ class WebhookService:
event_id=webhook.get_event_id() event_id=webhook.get_event_id()
) )
# 4. 获取项目映射和任务名 # 4. Get project mapping and job name
job_name = await self._determine_job_name(repository, branch) job_name = await self._determine_job_name(repository, branch)
if not job_name: if not job_name:
return WebhookResponse( return WebhookResponse(
@ -81,10 +78,10 @@ class WebhookService:
event_id=webhook.get_event_id() event_id=webhook.get_event_id()
) )
# 5. 准备任务参数 # 5. Prepare job parameters
job_params = self._prepare_job_parameters(webhook, job_name) job_params = self._prepare_job_parameters(webhook, job_name)
# 6. 提交任务到队列 # 6. Submit job to queue
task_result = await self._submit_job_to_queue( task_result = await self._submit_job_to_queue(
webhook, job_name, job_params webhook, job_name, job_params
) )
@ -114,13 +111,13 @@ class WebhookService:
) )
async def _determine_job_name(self, repository: str, branch: str) -> Optional[str]: async def _determine_job_name(self, repository: str, branch: str) -> Optional[str]:
"""根据仓库和分支确定任务名""" """Determine job name by repository and branch"""
# 首先尝试从数据库获取项目映射 # First try to get project mapping from database
job_name = await self.db_service.determine_job_name(repository, branch) job_name = await self.db_service.determine_job_name(repository, branch)
if job_name: if job_name:
return job_name return job_name
# 如果数据库中没有映射，使用配置文件中的环境分发 # If not found in database, use environment dispatch from config
environment = self.settings.get_environment_for_branch(branch) environment = self.settings.get_environment_for_branch(branch)
if environment: if environment:
return environment.jenkins_job return environment.jenkins_job
@ -128,7 +125,7 @@ class WebhookService:
return None return None
def _prepare_job_parameters(self, webhook: GiteaWebhook, job_name: str) -> Dict[str, str]: def _prepare_job_parameters(self, webhook: GiteaWebhook, job_name: str) -> Dict[str, str]:
"""准备 Jenkins 任务参数""" """Prepare Jenkins job parameters"""
author_info = webhook.get_author_info() author_info = webhook.get_author_info()
return { return {
@ -151,9 +148,9 @@ class WebhookService:
job_name: str, job_name: str,
job_params: Dict[str, str] job_params: Dict[str, str]
) -> bool: ) -> bool:
"""提交任务到 Celery 队列""" """Submit job to Celery queue"""
try: try:
# 创建任务 # Create task
task_kwargs = { task_kwargs = {
"job_name": job_name, "job_name": job_name,
"jenkins_url": self.settings.jenkins.url, "jenkins_url": self.settings.jenkins.url,
@ -162,14 +159,14 @@ class WebhookService:
"repository": webhook.repository.full_name, "repository": webhook.repository.full_name,
"branch": webhook.get_branch_name(), "branch": webhook.get_branch_name(),
"commit_hash": webhook.get_commit_hash(), "commit_hash": webhook.get_commit_hash(),
"priority": 1 # 默认优先级 "priority": 1 # Default priority
} }
# 提交到 Celery 队列 # Submit to Celery queue
task = self.celery_app.send_task( task = self.celery_app.send_task(
"app.tasks.jenkins_tasks.trigger_jenkins_job", "app.tasks.jenkins_tasks.trigger_jenkins_job",
kwargs=task_kwargs, kwargs=task_kwargs,
priority=environment.priority priority=task_kwargs["priority"]
) )
logger.info("Job submitted to queue", logger.info("Job submitted to queue",
@ -187,15 +184,15 @@ class WebhookService:
return False return False
async def get_webhook_stats(self) -> Dict[str, Any]: async def get_webhook_stats(self) -> Dict[str, Any]:
"""获取 Webhook 处理统计""" """Get webhook processing statistics"""
try: try:
# 获取队列统计 # Get queue stats
queue_stats = await self._get_queue_stats() queue_stats = await self._get_queue_stats()
# 获取防抖统计 # Get deduplication stats
dedup_stats = await self.dedup_service.get_stats() dedup_stats = await self.dedup_service.get_stats()
# 获取环境配置 # Get environment config
environments = {} environments = {}
for name, config in self.settings.environments.items(): for name, config in self.settings.environments.items():
environments[name] = { environments[name] = {
@ -222,20 +219,20 @@ class WebhookService:
return {"error": str(e)} return {"error": str(e)}
async def _get_queue_stats(self) -> Dict[str, Any]: async def _get_queue_stats(self) -> Dict[str, Any]:
"""获取队列统计信息""" """Get queue statistics"""
try: try:
# 获取 Celery 队列统计 # Get Celery queue stats
inspect = self.celery_app.control.inspect() inspect = self.celery_app.control.inspect()
# 活跃任务 # Active tasks
active = inspect.active() active = inspect.active()
active_count = sum(len(tasks) for tasks in active.values()) if active else 0 active_count = sum(len(tasks) for tasks in active.values()) if active else 0
# 等待任务 # Reserved tasks
reserved = inspect.reserved() reserved = inspect.reserved()
reserved_count = sum(len(tasks) for tasks in reserved.values()) if reserved else 0 reserved_count = sum(len(tasks) for tasks in reserved.values()) if reserved else 0
# 注册的 worker # Registered workers
registered = inspect.registered() registered = inspect.registered()
worker_count = len(registered) if registered else 0 worker_count = len(registered) if registered else 0
@ -251,9 +248,9 @@ class WebhookService:
return {"error": str(e)} return {"error": str(e)}
async def clear_queue(self) -> Dict[str, Any]: async def clear_queue(self) -> Dict[str, Any]:
"""清空队列""" """Clear queue"""
try: try:
# 撤销所有活跃任务 # Revoke all active tasks
inspect = self.celery_app.control.inspect() inspect = self.celery_app.control.inspect()
active = inspect.active() active = inspect.active()

85
apps/gitea-webhook-ambassador-python/app/tasks/jenkins_tasks.py
View File

@ -1,6 +1,6 @@
""" """
Jenkins 任务处理 Jenkins task processing
使用 Celery 处理异步 Jenkins 任务触发 Asynchronous Jenkins job triggering using Celery
""" """
import asyncio import asyncio
@ -17,7 +17,7 @@ from app.services.jenkins_service import JenkinsService
logger = structlog.get_logger() logger = structlog.get_logger()
settings = get_settings() settings = get_settings()
# 创建 Celery 应用 # Create Celery app
celery_app = Celery( celery_app = Celery(
"gitea_webhook_ambassador", "gitea_webhook_ambassador",
broker=settings.redis.url, broker=settings.redis.url,
@ -25,7 +25,7 @@ celery_app = Celery(
include=["app.tasks.jenkins_tasks"] include=["app.tasks.jenkins_tasks"]
) )
# Celery 配置 # Celery configuration
celery_app.conf.update( celery_app.conf.update(
task_serializer="json", task_serializer="json",
accept_content=["json"], accept_content=["json"],
@ -33,20 +33,20 @@ celery_app.conf.update(
timezone="UTC", timezone="UTC",
enable_utc=True, enable_utc=True,
task_track_started=True, task_track_started=True,
task_time_limit=300, # 5分钟超时 task_time_limit=300, # 5 minutes timeout
task_soft_time_limit=240, # 4分钟软超时 task_soft_time_limit=240, # 4 minutes soft timeout
worker_prefetch_multiplier=1, worker_prefetch_multiplier=1,
worker_max_tasks_per_child=1000, worker_max_tasks_per_child=1000,
worker_max_memory_per_child=200000, # 200MB worker_max_memory_per_child=200000, # 200MB
task_acks_late=True, task_acks_late=True,
task_reject_on_worker_lost=True, task_reject_on_worker_lost=True,
task_always_eager=False, # 生产环境设为 False task_always_eager=False, # Set to False in production
result_expires=3600, # 结果缓存1小时 result_expires=3600, # Result cache 1 hour
) )
class JenkinsTask(Task): class JenkinsTask(Task):
"""Jenkins 任务基类""" """Jenkins task base class"""
abstract = True abstract = True
@ -59,7 +59,7 @@ class JenkinsTask(Task):
return self.run(*args, **kwargs) return self.run(*args, **kwargs)
def on_failure(self, exc, task_id, args, kwargs, einfo): def on_failure(self, exc, task_id, args, kwargs, einfo):
"""任务失败回调""" """Task failure callback"""
logger.error("Task failed", logger.error("Task failed",
task_id=task_id, task_id=task_id,
task_name=self.name, task_name=self.name,
@ -68,7 +68,7 @@ class JenkinsTask(Task):
kwargs=kwargs) kwargs=kwargs)
def on_retry(self, exc, task_id, args, kwargs, einfo): def on_retry(self, exc, task_id, args, kwargs, einfo):
"""任务重试回调""" """Task retry callback"""
logger.warning("Task retrying", logger.warning("Task retrying",
task_id=task_id, task_id=task_id,
task_name=self.name, task_name=self.name,
@ -76,7 +76,7 @@ class JenkinsTask(Task):
retry_count=self.request.retries) retry_count=self.request.retries)
def on_success(self, retval, task_id, args, kwargs): def on_success(self, retval, task_id, args, kwargs):
"""任务成功回调""" """Task success callback"""
logger.info("Task completed successfully", logger.info("Task completed successfully",
task_id=task_id, task_id=task_id,
task_name=self.name, task_name=self.name,
@ -104,20 +104,18 @@ def trigger_jenkins_job(
priority: int = 1 priority: int = 1
) -> Dict[str, Any]: ) -> Dict[str, Any]:
""" """
触发 Jenkins 任务 Trigger Jenkins job
Args: Args:
job_name: Jenkins 任务名 job_name: Jenkins job name
jenkins_url: Jenkins URL jenkins_url: Jenkins URL
parameters: 任务参数 parameters: job parameters
event_id: 事件 ID event_id: event ID
repository: 仓库名 repository: repository name
branch: 分支名 branch: branch name
commit_hash: 提交哈希 commit_hash: commit hash
priority: 优先级 priority: priority
Returns: Returns:
Dict: 任务执行结果 Dict: job execution result
""" """
start_time = time.time() start_time = time.time()
@ -131,10 +129,10 @@ def trigger_jenkins_job(
commit_hash=commit_hash, commit_hash=commit_hash,
priority=priority) priority=priority)
# 创建 Jenkins 服务实例 # Create Jenkins service instance
jenkins_service = JenkinsService() jenkins_service = JenkinsService()
# 触发 Jenkins 任务 # Trigger Jenkins job
result = asyncio.run(jenkins_service.trigger_job( result = asyncio.run(jenkins_service.trigger_job(
job_name=job_name, job_name=job_name,
jenkins_url=jenkins_url, jenkins_url=jenkins_url,
@ -171,7 +169,7 @@ def trigger_jenkins_job(
error=result.get("error"), error=result.get("error"),
execution_time=execution_time) execution_time=execution_time)
# 重试任务 # Retry task
raise self.retry( raise self.retry(
countdown=settings.queue.retry_delay * (2 ** self.request.retries), countdown=settings.queue.retry_delay * (2 ** self.request.retries),
max_retries=settings.queue.max_retries max_retries=settings.queue.max_retries
@ -185,7 +183,7 @@ def trigger_jenkins_job(
error=str(e), error=str(e),
execution_time=execution_time) execution_time=execution_time)
# 重试任务 # Retry task
raise self.retry( raise self.retry(
countdown=settings.queue.retry_delay * (2 ** self.request.retries), countdown=settings.queue.retry_delay * (2 ** self.request.retries),
max_retries=settings.queue.max_retries max_retries=settings.queue.max_retries
@ -203,13 +201,11 @@ def check_jenkins_health(
jenkins_url: str jenkins_url: str
) -> Dict[str, Any]: ) -> Dict[str, Any]:
""" """
检查 Jenkins 健康状态 Check Jenkins health status
Args: Args:
jenkins_url: Jenkins URL jenkins_url: Jenkins URL
Returns: Returns:
Dict: 健康检查结果 Dict: health check result
""" """
try: try:
logger.info("Checking Jenkins health", jenkins_url=jenkins_url) logger.info("Checking Jenkins health", jenkins_url=jenkins_url)
@ -244,23 +240,22 @@ def check_jenkins_health(
) )
def cleanup_expired_tasks(self) -> Dict[str, Any]: def cleanup_expired_tasks(self) -> Dict[str, Any]:
""" """
清理过期任务 Clean up expired tasks
Returns: Returns:
Dict: 清理结果 Dict: cleanup result
""" """
try: try:
logger.info("Starting task cleanup") logger.info("Starting task cleanup")
# 获取所有任务 # Get all tasks
inspect = self.app.control.inspect() inspect = self.app.control.inspect()
# 清理过期的结果 # Clean up expired results
cleaned_count = 0 cleaned_count = 0
current_time = time.time() current_time = time.time()
# 这里可以添加更复杂的清理逻辑 # Add more complex cleanup logic here if needed
# 比如清理超过一定时间的任务结果 # For example, clean up results older than a certain time
logger.info("Task cleanup completed", cleaned_count=cleaned_count) logger.info("Task cleanup completed", cleaned_count=cleaned_count)
@ -280,27 +275,27 @@ def cleanup_expired_tasks(self) -> Dict[str, Any]:
} }
# 定时任务 # Periodic tasks
@celery_app.on_after_configure.connect @celery_app.on_after_configure.connect
def setup_periodic_tasks(sender, **kwargs): def setup_periodic_tasks(sender, **kwargs):
"""设置定时任务""" """Set up periodic tasks"""
# 每小时清理过期任务 # Clean up expired tasks every hour
sender.add_periodic_task( sender.add_periodic_task(
3600.0, # 1小时 3600.0, # 1 hour
cleanup_expired_tasks.s(), cleanup_expired_tasks.s(),
name="cleanup-expired-tasks" name="cleanup-expired-tasks"
) )
# 每5分钟检查 Jenkins 健康状态 # Check Jenkins health every 5 minutes
for env_name, env_config in settings.environments.items(): for env_name, env_config in settings.environments.items():
sender.add_periodic_task( sender.add_periodic_task(
300.0, # 5分钟 300.0, # 5 minutes
check_jenkins_health.s(env_config.jenkins_url), check_jenkins_health.s(env_config.jenkins_url),
name=f"check-jenkins-health-{env_name}" name=f"check-jenkins-health-{env_name}"
) )
def get_celery_app() -> Celery: def get_celery_app() -> Celery:
"""获取 Celery 应用实例""" """Get Celery app instance"""
return celery_app return celery_app

96
apps/gitea-webhook-ambassador-python/check_version.sh
View File

@ -1,104 +1,104 @@
#!/bin/bash #!/bin/bash
# Gitea Webhook Ambassador - 版本检查脚本 # Gitea Webhook Ambassador - Version Check Script
# 用于区分 Go 版本和 Python 版本 # Used to distinguish between Go and Python versions
echo "🔍 检查 Gitea Webhook Ambassador 版本..." echo "🔍 Checking Gitea Webhook Ambassador version..."
# 检查端口 8000 (Python 版本默认端口) # Check port 8000 (Python version default port)
echo "📡 检查端口 8000 (Python 版本)..." echo "📡 Checking port 8000 (Python version)..."
if lsof -i :8000 > /dev/null 2>&1; then if lsof -i :8000 > /dev/null 2>&1; then
PID=$(lsof -ti :8000) PID=$(lsof -ti :8000)
PROCESS=$(ps -p $PID -o comm= 2>/dev/null) PROCESS=$(ps -p $PID -o comm= 2>/dev/null)
echo "✅ 端口 8000 被占用 (PID: $PID, 进程: $PROCESS)" echo "✅ Port 8000 is occupied (PID: $PID, process: $PROCESS)"
# 检查是否是 Python 进程 # Check if it is a Python process
if echo "$PROCESS" | grep -q "python\|uvicorn"; then if echo "$PROCESS" | grep -q "python\|uvicorn"; then
echo "🐍 检测到 Python 版本正在运行" echo "🐍 Detected Python version is running"
# 尝试访问 Python 版本的 API # Try to access Python version API
if curl -s http://localhost:8000/api/health > /dev/null 2>&1; then if curl -s http://localhost:8000/api/health > /dev/null 2>&1; then
echo "✅ Python 版本 API 响应正常" echo "✅ Python version API is responsive"
echo "🌐 访问地址: http://localhost:8000" echo "🌐 Access: http://localhost:8000"
echo "📊 仪表板: http://localhost:8000/dashboard" echo "📊 Dashboard: http://localhost:8000/dashboard"
else else
echo "⚠️ Python 版本进程存在但 API 无响应" echo "⚠️ Python process exists but API is not responsive"
fi fi
else else
echo "⚠️ 端口 8000 被其他进程占用" echo "⚠️ Port 8000 is occupied by another process"
fi fi
else else
echo "❌ 端口 8000 未被占用 (Python 版本未运行)" echo "❌ Port 8000 is not occupied (Python version not running)"
fi fi
echo "" echo ""
# 检查端口 8080 (Go 版本默认端口) # Check port 8080 (Go version default port)
echo "📡 检查端口 8080 (Go 版本)..." echo "📡 Checking port 8080 (Go version)..."
if lsof -i :8080 > /dev/null 2>&1; then if lsof -i :8080 > /dev/null 2>&1; then
PID=$(lsof -ti :8080) PID=$(lsof -ti :8080)
PROCESS=$(ps -p $PID -o comm= 2>/dev/null) PROCESS=$(ps -p $PID -o comm= 2>/dev/null)
echo "✅ 端口 8080 被占用 (PID: $PID, 进程: $PROCESS)" echo "✅ Port 8080 is occupied (PID: $PID, process: $PROCESS)"
# 检查是否是 Go 进程 # Check if it is a Go process
if echo "$PROCESS" | grep -q "gitea-webhook-ambassador"; then if echo "$PROCESS" | grep -q "gitea-webhook-ambassador"; then
echo "🚀 检测到 Go 版本正在运行" echo "🚀 Detected Go version is running"
# 尝试访问 Go 版本的 API # Try to access Go version API
if curl -s http://localhost:8080/health > /dev/null 2>&1; then if curl -s http://localhost:8080/health > /dev/null 2>&1; then
echo "✅ Go 版本 API 响应正常" echo "✅ Go version API is responsive"
echo "🌐 访问地址: http://localhost:8080" echo "🌐 Access: http://localhost:8080"
else else
echo "⚠️ Go 版本进程存在但 API 无响应" echo "⚠️ Go process exists but API is not responsive"
fi fi
else else
echo "⚠️ 端口 8080 被其他进程占用" echo "⚠️ Port 8080 is occupied by another process"
fi fi
else else
echo "❌ 端口 8080 未被占用 (Go 版本未运行)" echo "❌ Port 8080 is not occupied (Go version not running)"
fi fi
echo "" echo ""
# 检查 PID 文件 # Check PID file
echo "📁 检查 PID 文件..." echo "📁 Checking PID file..."
# Python 版本 PID 文件 # Python version PID file
PYTHON_PID_FILE="/home/nicolas/freeleaps-ops/apps/gitea-webhook-ambassador-python/service.pid" PYTHON_PID_FILE="/home/nicolas/freeleaps-ops/apps/gitea-webhook-ambassador-python/service.pid"
if [ -f "$PYTHON_PID_FILE" ]; then if [ -f "$PYTHON_PID_FILE" ]; then
PYTHON_PID=$(cat "$PYTHON_PID_FILE") PYTHON_PID=$(cat "$PYTHON_PID_FILE")
if ps -p $PYTHON_PID > /dev/null 2>&1; then if ps -p $PYTHON_PID > /dev/null 2>&1; then
echo "✅ Python 版本 PID 文件存在 (PID: $PYTHON_PID)" echo "✅ Python version PID file exists (PID: $PYTHON_PID)"
else else
echo "⚠️ Python 版本 PID 文件存在但进程不存在" echo "⚠️ Python version PID file exists but process does not exist"
fi fi
else else
echo "❌ Python 版本 PID 文件不存在" echo "❌ Python version PID file does not exist"
fi fi
# Go 版本 PID 文件 (如果存在) # Go version PID file (if exists)
GO_PID_FILE="/home/nicolas/freeleaps-ops/apps/gitea-webhook-ambassador/service.pid" GO_PID_FILE="/home/nicolas/freeleaps-ops/apps/gitea-webhook-ambassador/service.pid"
if [ -f "$GO_PID_FILE" ]; then if [ -f "$GO_PID_FILE" ]; then
GO_PID=$(cat "$GO_PID_FILE") GO_PID=$(cat "$GO_PID_FILE")
if ps -p $GO_PID > /dev/null 2>&1; then if ps -p $GO_PID > /dev/null 2>&1; then
echo "✅ Go 版本 PID 文件存在 (PID: $GO_PID)" echo "✅ Go version PID file exists (PID: $GO_PID)"
else else
echo "⚠️ Go 版本 PID 文件存在但进程不存在" echo "⚠️ Go version PID file exists but process does not exist"
fi fi
else else
echo "❌ Go 版本 PID 文件不存在" echo "❌ Go version PID file does not exist"
fi fi
echo "" echo ""
# 总结 # Summary
echo "📊 总结:" echo "📊 Summary:"
echo "----------------------------------------" echo "----------------------------------------"
PYTHON_RUNNING=false PYTHON_RUNNING=false
GO_RUNNING=false GO_RUNNING=false
# 检查 Python 版本 # Check Python version
if lsof -i :8000 > /dev/null 2>&1; then if lsof -i :8000 > /dev/null 2>&1; then
PID=$(lsof -ti :8000) PID=$(lsof -ti :8000)
PROCESS=$(ps -p $PID -o comm= 2>/dev/null) PROCESS=$(ps -p $PID -o comm= 2>/dev/null)
@ -107,7 +107,7 @@ if lsof -i :8000 > /dev/null 2>&1; then
fi fi
fi fi
# 检查 Go 版本 # Check Go version
if lsof -i :8080 > /dev/null 2>&1; then if lsof -i :8080 > /dev/null 2>&1; then
PID=$(lsof -ti :8080) PID=$(lsof -ti :8080)
PROCESS=$(ps -p $PID -o comm= 2>/dev/null) PROCESS=$(ps -p $PID -o comm= 2>/dev/null)
@ -117,17 +117,17 @@ if lsof -i :8080 > /dev/null 2>&1; then
fi fi
if [ "$PYTHON_RUNNING" = true ] && [ "$GO_RUNNING" = true ]; then if [ "$PYTHON_RUNNING" = true ] && [ "$GO_RUNNING" = true ]; then
echo "⚠️ 两个版本都在运行！" echo "⚠️ Both versions are running!"
echo "🐍 Python 版本: http://localhost:8000" echo "🐍 Python version: http://localhost:8000"
echo "🚀 Go 版本: http://localhost:8080" echo "🚀 Go version: http://localhost:8080"
elif [ "$PYTHON_RUNNING" = true ]; then elif [ "$PYTHON_RUNNING" = true ]; then
echo "✅ 当前运行: Python 版本" echo "✅ Currently running: Python version"
echo "🌐 访问地址: http://localhost:8000" echo "🌐 Access: http://localhost:8000"
elif [ "$GO_RUNNING" = true ]; then elif [ "$GO_RUNNING" = true ]; then
echo "✅ 当前运行: Go 版本" echo "✅ Currently running: Go version"
echo "🌐 访问地址: http://localhost:8080" echo "🌐 Access: http://localhost:8080"
else else
echo "❌ 没有检测到任何版本在运行" echo "❌ No version is running"
fi fi
echo "----------------------------------------" echo "----------------------------------------"

12
apps/gitea-webhook-ambassador-python/config/environments.yaml
View File

@ -1,4 +1,4 @@
# 环境分发配置 # Environment dispatch configuration
environments: environments:
dev: dev:
branches: ["dev", "develop", "development", "feature/*"] branches: ["dev", "develop", "development", "feature/*"]
@ -24,16 +24,16 @@ environments:
jenkins_url: "https://jenkins-default.freeleaps.com" jenkins_url: "https://jenkins-default.freeleaps.com"
priority: 4 priority: 4
# 防抖配置 # Deduplication configuration
deduplication: deduplication:
enabled: true enabled: true
window_seconds: 300 # 5分钟防抖窗口 window_seconds: 300 # 5-minute deduplication window
strategy: "commit_branch" # commit_hash + branch strategy: "commit_branch" # commit_hash + branch
cache_ttl: 3600 # 缓存1小时 cache_ttl: 3600 # Cache for 1 hour
# 队列配置 # Queue configuration
queue: queue:
max_concurrent: 10 max_concurrent: 10
max_retries: 3 max_retries: 3
retry_delay: 60 # 秒 retry_delay: 60 # seconds
priority_levels: 4 priority_levels: 4

14
apps/gitea-webhook-ambassador-python/docker-compose.yml
View File

@ -1,7 +1,7 @@
version: '3.8' version: '3.8'
services: services:
# Redis 服务 # Redis service
redis: redis:
image: redis:7-alpine image: redis:7-alpine
container_name: webhook-ambassador-redis container_name: webhook-ambassador-redis
@ -16,7 +16,7 @@ services:
timeout: 10s timeout: 10s
retries: 3 retries: 3
# PostgreSQL 数据库 (可选，用于生产环境) # PostgreSQL database (optional, for production)
postgres: postgres:
image: postgres:15-alpine image: postgres:15-alpine
container_name: webhook-ambassador-postgres container_name: webhook-ambassador-postgres
@ -34,7 +34,7 @@ services:
timeout: 10s timeout: 10s
retries: 3 retries: 3
# Webhook Ambassador API 服务 # Webhook Ambassador API service
api: api:
build: build:
context: . context: .
@ -88,7 +88,7 @@ services:
condition: service_healthy condition: service_healthy
restart: unless-stopped restart: unless-stopped
# Celery Beat (定时任务调度器) # Celery Beat (scheduler)
beat: beat:
build: build:
context: . context: .
@ -112,7 +112,7 @@ services:
condition: service_healthy condition: service_healthy
restart: unless-stopped restart: unless-stopped
# Flower (Celery 监控) # Flower (Celery monitoring)
flower: flower:
build: build:
context: . context: .
@ -135,7 +135,7 @@ services:
condition: service_healthy condition: service_healthy
restart: unless-stopped restart: unless-stopped
# Prometheus (监控) # Prometheus (monitoring)
prometheus: prometheus:
image: prom/prometheus:latest image: prom/prometheus:latest
container_name: webhook-ambassador-prometheus container_name: webhook-ambassador-prometheus
@ -153,7 +153,7 @@ services:
- '--web.enable-lifecycle' - '--web.enable-lifecycle'
restart: unless-stopped restart: unless-stopped
# Grafana (监控面板) # Grafana (monitoring dashboard)
grafana: grafana:
image: grafana/grafana:latest image: grafana/grafana:latest
container_name: webhook-ambassador-grafana container_name: webhook-ambassador-grafana

18
apps/gitea-webhook-ambassador-python/env.example
View File

@ -1,41 +1,41 @@
# 应用配置 # Application configuration
APP_NAME=Gitea Webhook Ambassador APP_NAME=Gitea Webhook Ambassador
DEBUG=false DEBUG=false
HOST=0.0.0.0 HOST=0.0.0.0
PORT=8000 PORT=8000
# 数据库配置 # Database configuration
DATABASE_URL=sqlite:///./webhook_ambassador.db DATABASE_URL=sqlite:///./webhook_ambassador.db
# 生产环境使用 PostgreSQL: # For production, use PostgreSQL:
# DATABASE_URL=postgresql://webhook_user:webhook_password@localhost:5432/webhook_ambassador # DATABASE_URL=postgresql://webhook_user:webhook_password@localhost:5432/webhook_ambassador
# Redis 配置 # Redis configuration
REDIS_URL=redis://localhost:6379/0 REDIS_URL=redis://localhost:6379/0
REDIS_PASSWORD= REDIS_PASSWORD=
REDIS_DB=0 REDIS_DB=0
# Jenkins 配置 # Jenkins configuration
JENKINS_USERNAME=your_jenkins_username JENKINS_USERNAME=your_jenkins_username
JENKINS_TOKEN=115127e693f1bc6b7194f58ff6d6283bd0 JENKINS_TOKEN=115127e693f1bc6b7194f58ff6d6283bd0
JENKINS_TIMEOUT=30 JENKINS_TIMEOUT=30
# 安全配置 # Security configuration
SECURITY_SECRET_KEY=r6Y@QTb*7BQN@hDGsN SECURITY_SECRET_KEY=r6Y@QTb*7BQN@hDGsN
SECURITY_WEBHOOK_SECRET_HEADER=X-Gitea-Signature SECURITY_WEBHOOK_SECRET_HEADER=X-Gitea-Signature
SECURITY_RATE_LIMIT_PER_MINUTE=100 SECURITY_RATE_LIMIT_PER_MINUTE=100
# 日志配置 # Logging configuration
LOGGING_LEVEL=INFO LOGGING_LEVEL=INFO
LOGGING_FORMAT=json LOGGING_FORMAT=json
LOGGING_FILE= LOGGING_FILE=
# 队列配置 # Queue configuration
QUEUE_MAX_CONCURRENT=10 QUEUE_MAX_CONCURRENT=10
QUEUE_MAX_RETRIES=3 QUEUE_MAX_RETRIES=3
QUEUE_RETRY_DELAY=60 QUEUE_RETRY_DELAY=60
QUEUE_PRIORITY_LEVELS=3 QUEUE_PRIORITY_LEVELS=3
# 防抖配置 # Deduplication configuration
DEDUPLICATION_ENABLED=true DEDUPLICATION_ENABLED=true
DEDUPLICATION_WINDOW_SECONDS=300 DEDUPLICATION_WINDOW_SECONDS=300
DEDUPLICATION_STRATEGY=commit_branch DEDUPLICATION_STRATEGY=commit_branch

34
apps/gitea-webhook-ambassador-python/fix_pid.sh
View File

@ -1,41 +1,41 @@
#!/bin/bash #!/bin/bash
# 修复 PID 文件问题 # Fix PID file issue
echo "🔧 修复 PID 文件问题..." echo "🔧 Fixing PID file issue..."
# 查找 Python 服务进程 # Find Python service process
PID=$(lsof -ti :8000 2>/dev/null) PID=$(lsof -ti :8000 2>/dev/null)
if [ -n "$PID" ]; then if [ -n "$PID" ]; then
echo "✅ 找到运行中的 Python 服务 (PID: $PID)" echo "✅ Found running Python service (PID: $PID)"
# 检查进程是否是我们的服务 # Check if the process is our service
PROCESS=$(ps -p $PID -o comm= 2>/dev/null) PROCESS=$(ps -p $PID -o comm= 2>/dev/null)
if echo "$PROCESS" | grep -q "python"; then if echo "$PROCESS" | grep -q "python"; then
echo "🐍 确认是 Python 版本的 Gitea Webhook Ambassador" echo "🐍 Confirmed: Python version of Gitea Webhook Ambassador"
# 创建 PID 文件 # Create PID file
echo $PID > service.pid echo $PID > service.pid
echo "✅ 已创建 PID 文件: service.pid" echo "✅ PID file created: service.pid"
# 验证 PID 文件 # Verify PID file
if [ -f "service.pid" ]; then if [ -f "service.pid" ]; then
STORED_PID=$(cat service.pid) STORED_PID=$(cat service.pid)
echo "📝 PID 文件内容: $STORED_PID" echo "📝 PID file content: $STORED_PID"
if [ "$STORED_PID" = "$PID" ]; then if [ "$STORED_PID" = "$PID" ]; then
echo "✅ PID 文件修复成功" echo "✅ PID file fixed successfully"
echo "💡 现在可以使用 './devbox stop' 来停止服务" echo "💡 Now you can use './devbox stop' to stop the service"
else else
echo "❌ PID 文件内容不匹配" echo "❌ PID file content does not match"
fi fi
else else
echo "❌ 无法创建 PID 文件" echo "❌ Failed to create PID file"
fi fi
else else
echo "⚠️ 端口 8000 被其他进程占用" echo "⚠️ Port 8000 is occupied by another process"
fi fi
else else
echo "❌ 没有找到运行中的 Python 服务" echo "❌ No running Python service found"
echo "💡 请先启动服务: './devbox start'" echo "💡 Please start the service first: './devbox start'"
fi fi

16
apps/gitea-webhook-ambassador-python/quick_check.sh
View File

@ -1,31 +1,31 @@
#!/bin/bash #!/bin/bash
# 快速检查当前运行的版本 # Quick check of the currently running version
echo "🔍 快速检查 Gitea Webhook Ambassador 版本..." echo "🔍 Quick check of Gitea Webhook Ambassador version..."
# 检查 Python 版本 (端口 8000) # Check Python version (port 8000)
if lsof -i :8000 > /dev/null 2>&1; then if lsof -i :8000 > /dev/null 2>&1; then
PID=$(lsof -ti :8000) PID=$(lsof -ti :8000)
PROCESS=$(ps -p $PID -o comm= 2>/dev/null) PROCESS=$(ps -p $PID -o comm= 2>/dev/null)
if echo "$PROCESS" | grep -q "python\|uvicorn"; then if echo "$PROCESS" | grep -q "python\|uvicorn"; then
echo "🐍 Python 版本正在运行 (PID: $PID)" echo "🐍 Python version is running (PID: $PID)"
echo "🌐 http://localhost:8000" echo "🌐 http://localhost:8000"
echo "📊 http://localhost:8000/dashboard" echo "📊 http://localhost:8000/dashboard"
exit 0 exit 0
fi fi
fi fi
# 检查 Go 版本 (端口 8080) # Check Go version (port 8080)
if lsof -i :8080 > /dev/null 2>&1; then if lsof -i :8080 > /dev/null 2>&1; then
PID=$(lsof -ti :8080) PID=$(lsof -ti :8080)
PROCESS=$(ps -p $PID -o comm= 2>/dev/null) PROCESS=$(ps -p $PID -o comm= 2>/dev/null)
if echo "$PROCESS" | grep -q "gitea-webhook-ambassador"; then if echo "$PROCESS" | grep -q "gitea-webhook-ambassador"; then
echo "🚀 Go 版本正在运行 (PID: $PID)" echo "🚀 Go version is running (PID: $PID)"
echo "🌐 http://localhost:8080" echo "🌐 http://localhost:8080"
exit 0 exit 0
fi fi
fi fi
echo "❌ 没有检测到任何版本在运行" echo "❌ No version is running"
echo "💡 使用 './devbox start' 启动 Python 版本" echo "💡 Use './devbox start' to start the Python version"

64
apps/gitea-webhook-ambassador-python/scripts/setup.sh
View File

@ -1,61 +1,55 @@
#!/bin/bash #!/bin/bash
# Gitea Webhook Ambassador 快速设置脚本 # Gitea Webhook Ambassador quick setup script
set -e echo "🚀 Starting Gitea Webhook Ambassador setup..."
echo "🚀 开始设置 Gitea Webhook Ambassador..." # Check Python version
python_version=$(python3 -V 2>&1 | awk '{print $2}')
# 检查 Python 版本 if [[ "$python_version" < "3.8" ]]; then
python_version=$(python3 --version 2>&1 | grep -oE '[0-9]+\.[0-9]+') echo "❌ Python 3.8 or higher is required, current version: $python_version"
required_version="3.8"
if [ "$(printf '%s\n' "$required_version" "$python_version" | sort -V | head -n1)" != "$required_version" ]; then
echo "❌ 需要 Python 3.8 或更高版本，当前版本: $python_version"
exit 1 exit 1
fi fi
echo "✅ Python 版本检查通过: $python_version" echo "✅ Python version check passed: $python_version"
# 创建虚拟环境 # Create virtual environment
if [ ! -d "venv" ]; then if [ ! -d "venv" ]; then
echo "📦 创建虚拟环境..." echo "📦 Creating virtual environment..."
python3 -m venv venv python3 -m venv venv
fi fi
# 激活虚拟环境 # Activate virtual environment
echo "🔧 激活虚拟环境..." echo "🔧 Activating virtual environment..."
source venv/bin/activate source venv/bin/activate
# 升级 pip # Upgrade pip
echo "⬆️ 升级 pip..." echo "⬆️ Upgrading pip..."
pip install --upgrade pip pip install --upgrade pip
# 安装依赖 # Install dependencies
echo "📚 安装依赖..." echo "📚 Installing dependencies..."
pip install -r requirements.txt pip install -r requirements.txt
# 创建配置文件 # Create config file
if [ ! -f ".env" ]; then if [ ! -f ".env" ]; then
echo "⚙️ 创建环境配置文件..." echo "⚙️ Creating environment config file..."
cp env.example .env cp env.example .env
echo "📝 请编辑 .env 文件，配置您的 Jenkins 凭据和其他设置" echo "📝 Please edit the .env file to configure your Jenkins credentials and other settings"
fi fi
# 创建日志目录 # Create logs directory
mkdir -p logs mkdir -p logs
# 创建数据库目录 # Create database directory
mkdir -p data mkdir -p data
echo "✅ 设置完成！" echo "✅ Setup complete!"
echo "" echo "📋 Next steps:"
echo "📋 下一步操作：" echo "1. Edit the .env file to configure Jenkins credentials"
echo "1. 编辑 .env 文件，配置 Jenkins 凭据" echo "2. Run: source venv/bin/activate"
echo "2. 运行: source venv/bin/activate" echo "3. Start Redis: docker run -d -p 6379:6379 redis:alpine"
echo "3. 启动 Redis: docker run -d -p 6379:6379 redis:alpine" echo "4. Start the service: python -m uvicorn app.main:app --reload"
echo "4. 启动服务: python -m uvicorn app.main:app --reload" echo "5. Start Celery worker: celery -A app.tasks.jenkins_tasks worker --loglevel=info"
echo "5. 启动 Celery worker: celery -A app.tasks.jenkins_tasks worker --loglevel=info" echo "🌐 Access: http://localhost:8000"
echo "" echo "📊 Monitoring dashboard: http://localhost:8000/health"
echo "🌐 访问地址: http://localhost:8000"
echo "📊 监控面板: http://localhost:8000/health"

77
apps/gitea-webhook-ambassador-python/scripts/start.sh
View File

@ -1,64 +1,65 @@
#!/bin/bash #!/bin/bash
# Gitea Webhook Ambassador 启动脚本 # Gitea Webhook Ambassador start script
set -e # Check virtual environment
# 检查虚拟环境
if [ ! -d "venv" ]; then if [ ! -d "venv" ]; then
echo "❌ 虚拟环境不存在，请先运行 ./scripts/setup.sh" echo "❌ Virtual environment does not exist, please run ./scripts/setup.sh first"
exit 1 exit 1
fi fi
# 激活虚拟环境 # Activate virtual environment
source venv/bin/activate source venv/bin/activate
# 检查环境文件 # Check environment file
if [ ! -f ".env" ]; then if [ ! -f ".env" ]; then
echo "❌ .env 文件不存在，请先运行 ./scripts/setup.sh" echo "❌ .env file does not exist, please run ./scripts/setup.sh first"
exit 1 exit 1
fi fi
# 检查 Redis 是否运行 # Check if Redis is running
if ! docker ps | grep -q redis; then if ! pgrep -f redis-server > /dev/null; then
echo "🐳 启动 Redis..." echo "🐳 Starting Redis..."
docker run -d --name webhook-redis -p 6379:6379 redis:alpine docker run -d -p 6379:6379 redis:alpine
sleep 3
fi fi
echo "🚀 启动 Gitea Webhook Ambassador..." # Start Gitea Webhook Ambassador
echo "🚀 Starting Gitea Webhook Ambassador..."
# 启动 API 服务 # Start API service
echo "🌐 启动 API 服务..." API_LOG="logs/api.log"
python -m uvicorn app.main:app --host 0.0.0.0 --port 8000 --reload & echo "🌐 Starting API service..."
nohup python -m uvicorn app.main:app --host 0.0.0.0 --port 8000 > $API_LOG 2>&1 &
API_PID=$! API_PID=$!
# 等待 API 服务启动 # Wait for API service to start
sleep 5 sleep 3
# 启动 Celery worker # Start Celery worker
echo "⚙️ 启动 Celery worker..." WORKER_LOG="logs/worker.log"
celery -A app.tasks.jenkins_tasks worker --loglevel=info --concurrency=4 & echo "⚙️ Starting Celery worker..."
nohup celery -A app.tasks.jenkins_tasks worker --loglevel=info > $WORKER_LOG 2>&1 &
WORKER_PID=$! WORKER_PID=$!
# 启动 Celery beat (定时任务) # Start Celery beat (scheduled tasks)
echo "⏰ 启动定时任务调度器..." BEAT_LOG="logs/beat.log"
celery -A app.tasks.jenkins_tasks beat --loglevel=info & echo "⏰ Starting scheduled task scheduler..."
nohup celery -A app.tasks.jenkins_tasks beat --loglevel=info > $BEAT_LOG 2>&1 &
BEAT_PID=$! BEAT_PID=$!
echo "✅ 所有服务已启动！" # All services started
echo "" sleep 2
echo "📊 服务状态：" echo "✅ All services started!"
echo "- API 服务: http://localhost:8000 (PID: $API_PID)" echo "📊 Service status:"
echo "- 健康检查: http://localhost:8000/health" echo "- API service: http://localhost:8000 (PID: $API_PID)"
echo "- 监控指标: http://localhost:8000/metrics" echo "- Health check: http://localhost:8000/health"
echo "- Celery Worker: PID $WORKER_PID" echo "- Metrics: http://localhost:8000/metrics"
echo "- Celery Beat: PID $BEAT_PID"
echo ""
echo "🛑 按 Ctrl+C 停止所有服务"
# 等待中断信号 # Wait for Ctrl+C to stop all services
trap 'echo "🛑 正在停止服务..."; kill $API_PID $WORKER_PID $BEAT_PID 2>/dev/null; exit 0' INT echo "🛑 Press Ctrl+C to stop all services"
# 等待所有后台进程 # Wait for interrupt signal
trap 'echo "🛑 Stopping services..."; kill $API_PID $WORKER_PID $BEAT_PID 2>/dev/null; exit 0' INT
# Wait for all background processes
wait wait

94
apps/gitea-webhook-ambassador-python/start.sh
View File

@ -1,120 +1,120 @@
#!/bin/bash #!/bin/bash
# Gitea Webhook Ambassador Python 启动脚本 # Gitea Webhook Ambassador Python Start Script
SCRIPT_DIR="$(cd "$(dirname "${BASH_SOURCE[0]}")" && pwd)" SCRIPT_DIR="$(cd "$(dirname "${BASH_SOURCE[0]}")" && pwd)"
SERVICE_NAME="gitea-webhook-ambassador-python" SERVICE_NAME="gitea-webhook-ambassador-python"
LOG_FILE="$SCRIPT_DIR/logs/service.log" LOG_FILE="$SCRIPT_DIR/logs/service.log"
PID_FILE="$SCRIPT_DIR/service.pid" PID_FILE="$SCRIPT_DIR/service.pid"
# 创建日志目录 # Create logs directory
mkdir -p "$SCRIPT_DIR/logs" mkdir -p "$SCRIPT_DIR/logs"
# 激活虚拟环境 # Activate virtual environment
source "$SCRIPT_DIR/venv/bin/activate" source "$SCRIPT_DIR/venv/bin/activate"
# 函数：启动服务 # Function: Start service
start_service() { start_service() {
echo "🚀 启动 $SERVICE_NAME..." echo "🚀 Starting $SERVICE_NAME..."
if [ -f "$PID_FILE" ]; then if [ -f "$PID_FILE" ]; then
PID=$(cat "$PID_FILE") PID=$(cat "$PID_FILE")
if ps -p $PID > /dev/null 2>&1; then if ps -p $PID > /dev/null 2>&1; then
echo "❌ 服务已在运行 (PID: $PID)" echo "❌ Service is already running (PID: $PID)"
return 1 return 1
else else
echo "⚠️ 发现过期的 PID 文件，正在清理..." echo "⚠️ Found stale PID file, cleaning up..."
rm -f "$PID_FILE" rm -f "$PID_FILE"
fi fi
fi fi
# 后台启动服务 # Start service in background
nohup python -m uvicorn app.main_demo:app --host 0.0.0.0 --port 8000 > "$LOG_FILE" 2>&1 & nohup python -m uvicorn app.main_demo:app --host 0.0.0.0 --port 8000 > "$LOG_FILE" 2>&1 &
PID=$! PID=$!
echo $PID > "$PID_FILE" echo $PID > "$PID_FILE"
# 等待服务启动 # Wait for service to start
sleep 3 sleep 3
if ps -p $PID > /dev/null 2>&1; then if ps -p $PID > /dev/null 2>&1; then
echo "✅ 服务启动成功 (PID: $PID)" echo "✅ Service started successfully (PID: $PID)"
echo "📝 日志文件: $LOG_FILE" echo "📝 Log file: $LOG_FILE"
echo "🌐 访问地址: http://localhost:8000" echo "🌐 Access: http://localhost:8000"
echo "🔑 演示密钥: demo_admin_key, demo_user_key" echo "🔑 Demo keys: demo_admin_key, demo_user_key"
else else
echo "❌ 服务启动失败" echo "❌ Service failed to start"
rm -f "$PID_FILE" rm -f "$PID_FILE"
return 1 return 1
fi fi
} }
# 函数：停止服务 # Function: Stop service
stop_service() { stop_service() {
echo "🛑 停止 $SERVICE_NAME..." echo "🛑 Stopping $SERVICE_NAME..."
if [ -f "$PID_FILE" ]; then if [ -f "$PID_FILE" ]; then
PID=$(cat "$PID_FILE") PID=$(cat "$PID_FILE")
if ps -p $PID > /dev/null 2>&1; then if ps -p $PID > /dev/null 2>&1; then
kill $PID kill $PID
echo "✅ 服务已停止 (PID: $PID)" echo "✅ Service stopped (PID: $PID)"
else else
echo "⚠️ 服务未运行" echo "⚠️ Service not running"
fi fi
rm -f "$PID_FILE" rm -f "$PID_FILE"
else else
echo "⚠️ PID 文件不存在" echo "⚠️ PID file does not exist"
fi fi
} }
# 函数：重启服务 # Function: Restart service
restart_service() { restart_service() {
echo "🔄 重启 $SERVICE_NAME..." echo "🔄 Restarting $SERVICE_NAME..."
stop_service stop_service
sleep 2 sleep 2
start_service start_service
} }
# 函数：查看状态 # Function: Show status
status_service() { status_service() {
if [ -f "$PID_FILE" ]; then if [ -f "$PID_FILE" ]; then
PID=$(cat "$PID_FILE") PID=$(cat "$PID_FILE")
if ps -p $PID > /dev/null 2>&1; then if ps -p $PID > /dev/null 2>&1; then
echo "✅ $SERVICE_NAME 正在运行 (PID: $PID)" echo "✅ $SERVICE_NAME is running (PID: $PID)"
echo "📝 日志文件: $LOG_FILE" echo "📝 Log file: $LOG_FILE"
echo "🌐 访问地址: http://localhost:8000" echo "🌐 Access: http://localhost:8000"
else else
echo "❌ $SERVICE_NAME 未运行 (PID 文件存在但进程不存在)" echo "❌ $SERVICE_NAME is not running (PID file exists but process not found)"
rm -f "$PID_FILE" rm -f "$PID_FILE"
fi fi
else else
echo "❌ $SERVICE_NAME 未运行" echo "❌ $SERVICE_NAME is not running"
fi fi
} }
# 函数：查看日志 # Function: Show logs
show_logs() { show_logs() {
if [ -f "$LOG_FILE" ]; then if [ -f "$LOG_FILE" ]; then
echo "📝 显示最新日志 (最后 50 行):" echo "📝 Showing latest logs (last 50 lines):"
echo "----------------------------------------" echo "----------------------------------------"
tail -n 50 "$LOG_FILE" tail -n 50 "$LOG_FILE"
echo "----------------------------------------" echo "----------------------------------------"
echo "完整日志文件: $LOG_FILE" echo "Full log file: $LOG_FILE"
else else
echo "❌ 日志文件不存在" echo "❌ Log file does not exist"
fi fi
} }
# 函数：实时日志 # Function: Follow logs
follow_logs() { follow_logs() {
if [ -f "$LOG_FILE" ]; then if [ -f "$LOG_FILE" ]; then
echo "📝 实时日志 (按 Ctrl+C 退出):" echo "📝 Real-time logs (Ctrl+C to exit):"
tail -f "$LOG_FILE" tail -f "$LOG_FILE"
else else
echo "❌ 日志文件不存在" echo "❌ Log file does not exist"
fi fi
} }
# 主逻辑 # Main logic
case "$1" in case "$1" in
start) start)
start_service start_service
@ -135,20 +135,20 @@ case "$1" in
follow_logs follow_logs
;; ;;
*) *)
echo "用法: $0 {start|stop|restart|status|logs|follow}" echo "Usage: $0 {start|stop|restart|status|logs|follow}"
echo "" echo ""
echo "命令说明:" echo "Command description:"
echo " start - 启动服务" echo " start - Start service"
echo " stop - 停止服务" echo " stop - Stop service"
echo " restart - 重启服务" echo " restart - Restart service"
echo " status - 查看服务状态" echo " status - Show service status"
echo " logs - 查看最新日志" echo " logs - Show latest logs"
echo " follow - 实时查看日志" echo " follow - Show real-time logs"
echo "" echo ""
echo "示例:" echo "Examples:"
echo " $0 start # 启动服务" echo " $0 start # Start service"
echo " $0 status # 查看状态" echo " $0 status # Show status"
echo " $0 logs # 查看日志" echo " $0 logs # Show logs"
exit 1 exit 1
;; ;;
esac esac

150
apps/gitea-webhook-ambassador-python/test_auth.py
View File

@ -1,7 +1,7 @@
#!/usr/bin/env python3 #!/usr/bin/env python3
""" """
认证功能测试脚本 Authentication feature test script
演示如何正确使用 JWT 和 API 密钥认证 Demonstrates how to properly use JWT and API key authentication
""" """
import asyncio import asyncio
@ -11,140 +11,138 @@ from datetime import datetime
BASE_URL = "http://localhost:8000" BASE_URL = "http://localhost:8000"
async def test_jwt_authentication(): def print_divider():
"""测试 JWT 认证"""
print("🔐 测试 JWT 认证")
print("-" * 50) print("-" * 50)
# 注意：在实际应用中，JWT 令牌应该通过登录端点获取 async def test_jwt_authentication():
# 这里我们使用一个示例令牌（在实际环境中需要从登录端点获取） """Test JWT authentication"""
print("🔐 Testing JWT authentication")
print_divider()
# 模拟 JWT 令牌（实际应用中应该从登录端点获取） # Note: In actual applications, JWT tokens should be obtained via the login endpoint
# Here we use a sample token (in real environments, obtain from login endpoint)
# Simulate JWT token (should be obtained from login endpoint in real use)
jwt_token = "eyJhbGciOiJIUzI1NiIsInR5cCI6IkpXVCJ9.eyJzdWIiOiJhZG1pbiIsImV4cCI6MTczMjAwMDAwMH0.test" jwt_token = "eyJhbGciOiJIUzI1NiIsInR5cCI6IkpXVCJ9.eyJzdWIiOiJhZG1pbiIsImV4cCI6MTczMjAwMDAwMH0.test"
async with aiohttp.ClientSession() as session: async with aiohttp.ClientSession() as session:
# 使用 JWT 令牌访问管理端点 # Use JWT token to access admin endpoint
headers = {"Authorization": f"Bearer {jwt_token}"} headers = {"Authorization": f"Bearer {jwt_token}"}
# 测试访问日志端点 # Test access to logs endpoint
async with session.get(f"{BASE_URL}/api/logs", headers=headers) as response: async with session.get(f"{BASE_URL}/api/logs", headers=headers) as response:
if response.status == 200: if response.status == 200:
logs = await response.json() logs = await response.json()
print("✅ JWT 认证成功 - 日志访问") print("✅ JWT authentication succeeded - logs access")
print(f" 获取到 {len(logs)} 条日志") print(f" Retrieved {len(logs)} logs")
else: else:
print(f"❌ JWT 认证失败 - 日志访问: {response.status}") print(f"❌ JWT authentication failed - logs access: {response.status}")
if response.status == 401: if response.status == 401:
print(" 原因: JWT 令牌无效或已过期") print(" Reason: JWT token is invalid or expired")
print() print()
async def test_api_key_authentication(): async def test_api_key_authentication():
"""测试 API 密钥认证""" """Test API key authentication"""
print("🔑 测试 API 密钥认证") print("🔑 Testing API key authentication")
print("-" * 50) print_divider()
async with aiohttp.ClientSession() as session: async with aiohttp.ClientSession() as session:
# 首先创建一个 API 密钥（需要管理员权限） # First, create an API key (requires admin privileges)
# 注意：这里我们使用一个临时的认证方式 # Note: Here we use a temporary authentication method
# 方法1：直接使用内存中的 API 密钥（仅用于演示） # Method 1: Use in-memory API key (for demo only)
# 在实际应用中，API 密钥应该通过管理界面创建 # In real applications, API keys should be created via the admin interface
# 模拟一个有效的 API 密钥 # Simulate a valid API key
api_key = "test_api_key_12345" api_key = "test_api_key_12345"
headers = {"Authorization": f"Bearer {api_key}"} headers = {"Authorization": f"Bearer {api_key}"}
# 测试访问日志端点 # Test access to logs endpoint
async with session.get(f"{BASE_URL}/api/logs", headers=headers) as response: async with session.get(f"{BASE_URL}/api/logs", headers=headers) as response:
if response.status == 200: if response.status == 200:
logs = await response.json() logs = await response.json()
print("✅ API 密钥认证成功 - 日志访问") print("✅ API key authentication succeeded - logs access")
print(f" 获取到 {len(logs)} 条日志") print(f" Retrieved {len(logs)} logs")
else: else:
print(f"❌ API 密钥认证失败 - 日志访问: {response.status}") print(f"❌ API key authentication failed - logs access: {response.status}")
if response.status == 401: if response.status == 401:
print(" 原因: API 密钥无效或已撤销") print(" Reason: API key is invalid or revoked")
print() print()
async def test_public_endpoints(): async def test_public_endpoints():
"""测试公开端点（无需认证）""" """Test public endpoints (no authentication required)"""
print("🌐 测试公开端点") print("🌐 Testing public endpoints")
print("-" * 50) print_divider()
async with aiohttp.ClientSession() as session: async with aiohttp.ClientSession() as session:
# 健康检查端点（无需认证） # Health check endpoint (no authentication required)
async with session.get(f"{BASE_URL}/health") as response: async with session.get(f"{BASE_URL}/health") as response:
if response.status == 200: if response.status == 200:
data = await response.json() data = await response.json()
print("✅ 健康检查端点访问成功") print("✅ Health check endpoint accessed successfully")
print(f" 状态: {data['status']}") print(f" Status: {data['status']}")
print(f" Jenkins: {data['jenkins']['status']}")
else: else:
print(f"❌ 健康检查端点访问失败: {response.status}") print(f"❌ Health check endpoint access failed: {response.status}")
# Webhook 端点（无需认证） # Webhook endpoint (no authentication required)
webhook_data = {"test": "webhook_data"} webhook_data = {"test": "webhook_data"}
async with session.post(f"{BASE_URL}/webhook/gitea", json=webhook_data) as response: async with session.post(f"{BASE_URL}/webhook/gitea", json=webhook_data) as response:
if response.status == 200: if response.status == 200:
data = await response.json() data = await response.json()
print("✅ Webhook 端点访问成功") print("✅ Webhook endpoint accessed successfully")
print(f" 响应: {data['message']}") print(f" Response: {data['message']}")
else: else:
print(f"❌ Webhook 端点访问失败: {response.status}") print(f"❌ Webhook endpoint access failed: {response.status}")
print() print()
async def test_authentication_flow(): async def test_authentication_flow():
"""测试完整的认证流程""" """Test the complete authentication flow"""
print("🔄 测试完整认证流程") print("🔄 Testing complete authentication flow")
print("-" * 50) print_divider()
print("📋 认证流程说明:") print("📋 Authentication flow description:")
print("1. 公开端点: /health, /webhook/gitea - 无需认证") print("1. Public endpoints: /health, /webhook/gitea - no authentication required")
print("2. 管理端点: /api/admin/* - 需要 JWT 或 API 密钥") print("2. Admin endpoints: /api/admin/* - JWT or API key required")
print("3. 日志端点: /api/logs/* - 需要 JWT 或 API 密钥") print("3. Logs endpoints: /api/logs/* - JWT or API key required")
print() print()
print("🔧 如何获取认证令牌:") print("🔧 How to obtain authentication tokens:")
print("1. JWT 令牌: 通过登录端点获取（需要实现登录功能）") print("1. JWT token: Obtain via login endpoint (login feature required)")
print("2. API 密钥: 通过管理界面创建（需要管理员权限）") print("2. API key: Create via admin interface (admin privileges required)")
print() print()
print("⚠️ 当前演示限制:") print("⚠️ Demo limitations:")
print("- 使用模拟的认证令牌") print("- Using simulated authentication tokens")
print("- 实际应用中需要实现完整的登录和密钥管理") print("- In real applications, implement full login and key management")
print("- 建议在生产环境中使用真实的认证系统") print("- It is recommended to use real authentication systems in production")
print() print()
async def create_demo_api_key(): async def create_demo_api_key():
"""创建演示用的 API 密钥""" """Create a demo API key"""
print("🔧 创建演示 API 密钥") print("🔧 Creating demo API key")
print("-" * 50) print_divider()
# 注意：这是一个简化的演示 # Note: This is a simplified demo
# 在实际应用中，API 密钥应该通过安全的方式创建和存储 # In real applications, API keys should be created and stored securely
demo_api_key = "demo_api_key_" + str(int(datetime.now().timestamp())) demo_api_key = "demo_api_key_" + str(int(datetime.now().timestamp()))
print(f"✅ 演示 API 密钥已创建: {demo_api_key}") print(f"✅ Demo API key created: {demo_api_key}")
print("📝 使用方法:") print("📝 Usage:")
print(f" curl -H 'Authorization: Bearer {demo_api_key}' {BASE_URL}/api/logs") print(f" curl -H 'Authorization: Bearer {demo_api_key}' {BASE_URL}/api/logs")
print() print()
return demo_api_key return demo_api_key
async def main(): async def main():
"""主测试函数""" """Main test function"""
print("🚀 开始认证功能测试") print("🚀 Starting authentication feature tests")
print("=" * 60) print("=" * 60)
print() print()
try: try:
# 等待服务启动 # Wait for service to start
await asyncio.sleep(2) await asyncio.sleep(2)
await test_public_endpoints() await test_public_endpoints()
@ -152,21 +150,21 @@ async def main():
await test_api_key_authentication() await test_api_key_authentication()
await test_authentication_flow() await test_authentication_flow()
# 创建演示 API 密钥 # Create demo API key
demo_key = await create_demo_api_key() demo_key = await create_demo_api_key()
print("=" * 60) print("=" * 60)
print("🎉 认证功能测试完成！") print("🎉 Authentication feature tests completed!")
print() print()
print("📚 下一步建议:") print("📚 Next steps:")
print("1. 实现完整的登录系统") print("1. Implement a full login system")
print("2. 添加用户管理功能") print("2. Add user management features")
print("3. 实现 API 密钥的安全存储") print("3. Implement secure API key storage")
print("4. 添加权限控制机制") print("4. Add permission control mechanisms")
print("5. 实现会话管理") print("5. Implement session management")
except Exception as e: except Exception as e:
print(f"❌ 测试过程中出现错误: {str(e)}") print(f"❌ Error occurred during testing: {str(e)}")
if __name__ == "__main__": if __name__ == "__main__":
asyncio.run(main()) asyncio.run(main())

151
apps/gitea-webhook-ambassador-python/test_enhanced.py
View File

@ -1,7 +1,7 @@
#!/usr/bin/env python3 #!/usr/bin/env python3
""" """
增强版 Gitea Webhook Ambassador 功能测试脚本 Enhanced Gitea Webhook Ambassador feature test script
演示所有新增的监测和管理功能 Demonstrates all new monitoring and management features
""" """
import asyncio import asyncio
@ -11,32 +11,34 @@ from datetime import datetime
BASE_URL = "http://localhost:8000" BASE_URL = "http://localhost:8000"
async def test_health_check(): def print_divider():
"""测试增强的健康检查"""
print("🧪 测试增强的健康检查")
print("-" * 50) print("-" * 50)
async def test_health_check():
"""Test enhanced health check"""
print("🧪 Testing enhanced health check")
print_divider()
async with aiohttp.ClientSession() as session: async with aiohttp.ClientSession() as session:
async with session.get(f"{BASE_URL}/health") as response: async with session.get(f"{BASE_URL}/health") as response:
if response.status == 200: if response.status == 200:
data = await response.json() data = await response.json()
print("✅ 健康检查通过") print("✅ Health check passed")
print(f" 状态: {data['status']}") print(f" Status: {data['status']}")
print(f" 服务: {data['service']}") print(f" Service: {data['service']}")
print(f" Jenkins: {data['jenkins']['status']}") print(f" Jenkins: {data['jenkins']['status']}")
print(f" 工作池: {data['worker_pool']['active_workers']} 活跃工作线程") print(f" Worker pool: {data['worker_pool']['active_workers']} active workers")
print(f" 队列大小: {data['worker_pool']['queue_size']}") print(f" Queue size: {data['worker_pool']['queue_size']}")
print(f" 已处理: {data['worker_pool']['total_processed']}") print(f" Processed: {data['worker_pool']['total_processed']}")
print(f" 失败: {data['worker_pool']['total_failed']}") print(f" Failed: {data['worker_pool']['total_failed']}")
else: else:
print(f"❌ 健康检查失败: {response.status}") print(f"❌ Health check failed: {response.status}")
print() print()
async def test_webhook(): async def test_webhook():
"""测试 Webhook 功能""" """Test webhook feature"""
print("🧪 测试 Webhook 功能") print("🧪 Testing webhook feature")
print("-" * 50) print_divider()
webhook_data = { webhook_data = {
"ref": "refs/heads/dev", "ref": "refs/heads/dev",
@ -59,22 +61,21 @@ async def test_webhook():
) as response: ) as response:
if response.status == 200: if response.status == 200:
data = await response.json() data = await response.json()
print("✅ Webhook 处理成功") print("✅ Webhook processed successfully")
print(f" 响应: {data['message']}") print(f" Response: {data['message']}")
print(f" 数据大小: {data['data']['body_size']} bytes") print(f" Data size: {data['data']['body_size']} bytes")
else: else:
print(f"❌ Webhook 处理失败: {response.status}") print(f"❌ Webhook processing failed: {response.status}")
print() print()
async def test_api_key_management(): async def test_api_key_management():
"""测试 API 密钥管理""" """Test API key management"""
print("🧪 测试 API 密钥管理") print("🧪 Testing API key management")
print("-" * 50) print_divider()
# 创建 API 密钥 # Create API key
async with aiohttp.ClientSession() as session: async with aiohttp.ClientSession() as session:
# 创建密钥 # Create key
create_data = {"name": "test-api-key"} create_data = {"name": "test-api-key"}
async with session.post( async with session.post(
f"{BASE_URL}/api/admin/api-keys", f"{BASE_URL}/api/admin/api-keys",
@ -85,41 +86,40 @@ async def test_api_key_management():
data = await response.json() data = await response.json()
api_key = data['key'] api_key = data['key']
key_id = data['id'] key_id = data['id']
print(f"✅ API 密钥创建成功") print(f"✅ API key created successfully")
print(f" ID: {key_id}") print(f" ID: {key_id}")
print(f" 名称: {data['name']}") print(f" Name: {data['name']}")
print(f" 密钥: {api_key[:8]}...{api_key[-8:]}") print(f" Key: {api_key[:8]}...{api_key[-8:]}")
# 使用新创建的密钥测试日志端点 # Test logs endpoint with new key
print("\n 测试使用新密钥访问日志端点...") print("\n Testing logs endpoint with new key...")
async with session.get( async with session.get(
f"{BASE_URL}/api/logs", f"{BASE_URL}/api/logs",
headers={"Authorization": f"Bearer {api_key}"} headers={"Authorization": f"Bearer {api_key}"}
) as log_response: ) as log_response:
if log_response.status == 200: if log_response.status == 200:
logs = await log_response.json() logs = await log_response.json()
print(f" ✅ 日志访问成功，获取到 {len(logs)} 条日志") print(f" ✅ Logs access succeeded, retrieved {len(logs)} logs")
else: else:
print(f" ❌ 日志访问失败: {log_response.status}") print(f" ❌ Logs access failed: {log_response.status}")
# 删除密钥 # Delete key
async with session.delete( async with session.delete(
f"{BASE_URL}/api/admin/api-keys/{key_id}", f"{BASE_URL}/api/admin/api-keys/{key_id}",
headers={"Authorization": f"Bearer {api_key}"} headers={"Authorization": f"Bearer {api_key}"}
) as delete_response: ) as delete_response:
if delete_response.status == 200: if delete_response.status == 200:
print(f" ✅ API 密钥删除成功") print(f" ✅ API key deleted successfully")
else: else:
print(f" ❌ API 密钥删除失败: {delete_response.status}") print(f" ❌ API key deletion failed: {delete_response.status}")
else: else:
print(f"❌ API 密钥创建失败: {response.status}") print(f"❌ API key creation failed: {response.status}")
print() print()
async def test_project_mapping(): async def test_project_mapping():
"""测试项目映射管理""" """Test project mapping management"""
print("🧪 测试项目映射管理") print("🧪 Testing project mapping management")
print("-" * 50) print_divider()
mapping_data = { mapping_data = {
"repository_name": "freeleaps/test-project", "repository_name": "freeleaps/test-project",
@ -135,7 +135,7 @@ async def test_project_mapping():
} }
async with aiohttp.ClientSession() as session: async with aiohttp.ClientSession() as session:
# 创建项目映射 # Create project mapping
async with session.post( async with session.post(
f"{BASE_URL}/api/admin/projects", f"{BASE_URL}/api/admin/projects",
json=mapping_data, json=mapping_data,
@ -143,45 +143,43 @@ async def test_project_mapping():
) as response: ) as response:
if response.status == 200: if response.status == 200:
data = await response.json() data = await response.json()
print("✅ 项目映射创建成功") print("✅ Project mapping created successfully")
print(f" ID: {data['id']}") print(f" ID: {data['id']}")
print(f" 仓库: {data['repository_name']}") print(f" Repository: {data['repository_name']}")
print(f" 默认任务: {data['default_job']}") print(f" Default job: {data['default_job']}")
print(f" 分支任务数: {len(data['branch_jobs'])}") print(f" Branch jobs: {len(data['branch_jobs'])}")
print(f" 分支模式数: {len(data['branch_patterns'])}") print(f" Branch patterns: {len(data['branch_patterns'])}")
else: else:
print(f"❌ 项目映射创建失败: {response.status}") print(f"❌ Project mapping creation failed: {response.status}")
print() print()
async def test_logs_and_stats(): async def test_logs_and_stats():
"""测试日志和统计功能""" """Test logs and statistics features"""
print("🧪 测试日志和统计功能") print("🧪 Testing logs and statistics features")
print("-" * 50) print_divider()
async with aiohttp.ClientSession() as session: async with aiohttp.ClientSession() as session:
# 获取日志统计 # Get log statistics
async with session.get( async with session.get(
f"{BASE_URL}/api/logs/stats", f"{BASE_URL}/api/logs/stats",
headers={"Authorization": "Bearer test-token"} headers={"Authorization": "Bearer test-token"}
) as response: ) as response:
if response.status == 200: if response.status == 200:
stats = await response.json() stats = await response.json()
print("✅ 日志统计获取成功") print("✅ Log statistics retrieved successfully")
print(f" 总日志数: {stats['total_logs']}") print(f" Total logs: {stats['total_logs']}")
print(f" 成功日志: {stats['successful_logs']}") print(f" Successful logs: {stats['successful_logs']}")
print(f" 失败日志: {stats['failed_logs']}") print(f" Failed logs: {stats['failed_logs']}")
print(f" 最近24小时: {stats['recent_logs_24h']}") print(f" Recent logs (24h): {stats['recent_logs_24h']}")
print(f" 仓库统计: {len(stats['repository_stats'])} 个仓库") print(f" Repository stats: {len(stats['repository_stats'])} repositories")
else: else:
print(f"❌ 日志统计获取失败: {response.status}") print(f"❌ Log statistics retrieval failed: {response.status}")
print() print()
async def test_admin_stats(): async def test_admin_stats():
"""测试管理统计""" """Test admin statistics"""
print("🧪 测试管理统计") print("🧪 Testing admin statistics")
print("-" * 50) print_divider()
async with aiohttp.ClientSession() as session: async with aiohttp.ClientSession() as session:
async with session.get( async with session.get(
@ -190,19 +188,18 @@ async def test_admin_stats():
) as response: ) as response:
if response.status == 200: if response.status == 200:
stats = await response.json() stats = await response.json()
print("✅ 管理统计获取成功") print("✅ Admin statistics retrieved successfully")
print(f" API 密钥总数: {stats['api_keys']['total']}") print(f" Total API keys: {stats['api_keys']['total']}")
print(f" 活跃密钥: {stats['api_keys']['active']}") print(f" Active keys: {stats['api_keys']['active']}")
print(f" 最近使用: {stats['api_keys']['recently_used']}") print(f" Recently used: {stats['api_keys']['recently_used']}")
print(f" 项目映射总数: {stats['project_mappings']['total']}") print(f" Total project mappings: {stats['project_mappings']['total']}")
else: else:
print(f"❌ 管理统计获取失败: {response.status}") print(f"❌ Admin statistics retrieval failed: {response.status}")
print() print()
async def main(): async def main():
"""主测试函数""" """Main test function"""
print("🚀 开始增强版 Gitea Webhook Ambassador 功能测试") print("🚀 Starting enhanced Gitea Webhook Ambassador feature tests")
print("=" * 60) print("=" * 60)
print() print()
@ -215,11 +212,11 @@ async def main():
await test_admin_stats() await test_admin_stats()
print("=" * 60) print("=" * 60)
print("🎉 所有测试完成！") print("🎉 All tests completed!")
print("✅ Python 版本现在具备了与 Go 版本相同的监测和管理功能") print("✅ Python version now has the same monitoring and management features as the Go version")
except Exception as e: except Exception as e:
print(f"❌ 测试过程中出现错误: {str(e)}") print(f"❌ Error occurred during testing: {str(e)}")
if __name__ == "__main__": if __name__ == "__main__":
asyncio.run(main()) asyncio.run(main())

151
apps/gitea-webhook-ambassador-python/test_enhanced_features.py
View File

@ -1,6 +1,6 @@
#!/usr/bin/env python3 #!/usr/bin/env python3
""" """
Gitea Webhook Ambassador 增强版功能测试脚本 Gitea Webhook Ambassador Enhanced Feature Test Script
""" """
import requests import requests
@ -11,29 +11,29 @@ BASE_URL = "http://localhost:8000"
ADMIN_SECRET_KEY = "admin-secret-key-change-in-production" ADMIN_SECRET_KEY = "admin-secret-key-change-in-production"
def test_health_check(): def test_health_check():
"""测试健康检查""" """Test health check"""
print("🔍 测试健康检查...") print("🔍 Testing health check...")
try: try:
response = requests.get(f"{BASE_URL}/health") response = requests.get(f"{BASE_URL}/health")
if response.status_code == 200: if response.status_code == 200:
data = response.json() data = response.json()
print(f"✅ 健康检查成功: {data['status']}") print(f"✅ Health check succeeded: {data['status']}")
print(f" 版本: {data['version']}") print(f" Version: {data['version']}")
print(f" 运行时间: {data['uptime']}") print(f" Uptime: {data['uptime']}")
print(f" 内存使用: {data['memory']}") print(f" Memory usage: {data['memory']}")
return True return True
else: else:
print(f"❌ 健康检查失败: {response.status_code}") print(f"❌ Health check failed: {response.status_code}")
return False return False
except Exception as e: except Exception as e:
print(f"❌ 健康检查异常: {e}") print(f"❌ Health check exception: {e}")
return False return False
def test_login(): def test_login():
"""测试登录功能""" """Test login functionality"""
print("\n🔐 测试登录功能...") print("\n🔐 Testing login functionality...")
try: try:
# 测试登录 API # Test login API
login_data = {"secret_key": ADMIN_SECRET_KEY} login_data = {"secret_key": ADMIN_SECRET_KEY}
response = requests.post( response = requests.post(
f"{BASE_URL}/api/auth/login", f"{BASE_URL}/api/auth/login",
@ -44,23 +44,23 @@ def test_login():
if response.status_code == 200: if response.status_code == 200:
data = response.json() data = response.json()
token = data.get("token") token = data.get("token")
print(f"✅ 登录成功，获得 JWT 令牌") print(f"✅ Login succeeded, obtained JWT token")
return token return token
else: else:
print(f"❌ 登录失败: {response.status_code} - {response.text}") print(f"❌ Login failed: {response.status_code} - {response.text}")
return None return None
except Exception as e: except Exception as e:
print(f"❌ 登录异常: {e}") print(f"❌ Login exception: {e}")
return None return None
def test_api_key_management(token): def test_api_key_management(token):
"""测试 API 密钥管理""" """Test API key management"""
print("\n🔑 测试 API 密钥管理...") print("\n🔑 Testing API key management...")
headers = {"Authorization": f"Bearer {token}"} headers = {"Authorization": f"Bearer {token}"}
try: try:
# 创建 API 密钥 # Create API key
key_data = {"description": "测试 API 密钥"} key_data = {"description": "Test API key"}
response = requests.post( response = requests.post(
f"{BASE_URL}/api/keys", f"{BASE_URL}/api/keys",
json=key_data, json=key_data,
@ -71,41 +71,41 @@ def test_api_key_management(token):
data = response.json() data = response.json()
api_key = data["key"] api_key = data["key"]
key_id = data["id"] key_id = data["id"]
print(f"✅ 创建 API 密钥成功: {api_key[:20]}...") print(f"✅ API key created successfully: {api_key[:20]}...")
# 列出 API 密钥 # List API keys
response = requests.get(f"{BASE_URL}/api/keys", headers=headers) response = requests.get(f"{BASE_URL}/api/keys", headers=headers)
if response.status_code == 200: if response.status_code == 200:
keys_data = response.json() keys_data = response.json()
print(f"✅ 列出 API 密钥成功，共 {len(keys_data['keys'])} 个") print(f"✅ API keys listed successfully, total {len(keys_data['keys'])}")
# 删除 API 密钥 # Delete API key
response = requests.delete(f"{BASE_URL}/api/keys/{key_id}", headers=headers) response = requests.delete(f"{BASE_URL}/api/keys/{key_id}", headers=headers)
if response.status_code == 200: if response.status_code == 200:
print(f"✅ 删除 API 密钥成功") print(f"✅ API key deleted successfully")
return True return True
else: else:
print(f"❌ 删除 API 密钥失败: {response.status_code}") print(f"❌ API key deletion failed: {response.status_code}")
return False return False
else: else:
print(f"❌ 列出 API 密钥失败: {response.status_code}") print(f"❌ API key listing failed: {response.status_code}")
return False return False
else: else:
print(f"❌ 创建 API 密钥失败: {response.status_code} - {response.text}") print(f"❌ API key creation failed: {response.status_code} - {response.text}")
return False return False
except Exception as e: except Exception as e:
print(f"❌ API 密钥管理异常: {e}") print(f"❌ API key management exception: {e}")
return False return False
def test_project_management(token): def test_project_management(token):
"""测试项目管理""" """Test project management"""
print("\n📁 测试项目管理...") print("\n📁 Testing project management...")
headers = {"Authorization": f"Bearer {token}"} headers = {"Authorization": f"Bearer {token}"}
try: try:
# 创建项目 # Create project
project_data = { project_data = {
"name": "测试项目", "name": "Test Project",
"jenkinsJob": "test-job", "jenkinsJob": "test-job",
"giteaRepo": "test-owner/test-repo" "giteaRepo": "test-owner/test-repo"
} }
@ -118,109 +118,108 @@ def test_project_management(token):
if response.status_code == 200: if response.status_code == 200:
data = response.json() data = response.json()
project_id = data["id"] project_id = data["id"]
print(f"✅ 创建项目成功: {data['name']}") print(f"✅ Project created successfully: {data['name']}")
# 列出项目 # List projects
response = requests.get(f"{BASE_URL}/api/projects/", headers=headers) response = requests.get(f"{BASE_URL}/api/projects/", headers=headers)
if response.status_code == 200: if response.status_code == 200:
projects_data = response.json() projects_data = response.json()
print(f"✅ 列出项目成功，共 {len(projects_data['projects'])} 个") print(f"✅ Projects listed successfully, total {len(projects_data['projects'])}")
# 删除项目 # Delete project
response = requests.delete(f"{BASE_URL}/api/projects/{project_id}", headers=headers) response = requests.delete(f"{BASE_URL}/api/projects/{project_id}", headers=headers)
if response.status_code == 200: if response.status_code == 200:
print(f"✅ 删除项目成功") print(f"✅ Project deleted successfully")
return True return True
else: else:
print(f"❌ 删除项目失败: {response.status_code}") print(f"❌ Project deletion failed: {response.status_code}")
return False return False
else: else:
print(f"❌ 列出项目失败: {response.status_code}") print(f"❌ Project listing failed: {response.status_code}")
return False return False
else: else:
print(f"❌ 创建项目失败: {response.status_code} - {response.text}") print(f"❌ Project creation failed: {response.status_code} - {response.text}")
return False return False
except Exception as e: except Exception as e:
print(f"❌ 项目管理异常: {e}") print(f"❌ Project management exception: {e}")
return False return False
def test_stats(token): def test_stats(token):
"""测试统计信息""" """Test statistics information"""
print("\n📊 测试统计信息...") print("\n📊 Testing statistics information...")
headers = {"Authorization": f"Bearer {token}"} headers = {"Authorization": f"Bearer {token}"}
try: try:
response = requests.get(f"{BASE_URL}/api/stats", headers=headers) response = requests.get(f"{BASE_URL}/api/stats", headers=headers)
if response.status_code == 200: if response.status_code == 200:
data = response.json() data = response.json()
print(f"✅ 获取统计信息成功:") print(f"✅ Statistics retrieved successfully:")
print(f" 总项目数: {data['total_projects']}") print(f" Total projects: {data['total_projects']}")
print(f" API 密钥数: {data['total_api_keys']}") print(f" API keys: {data['total_api_keys']}")
print(f" 今日触发次数: {data['today_triggers']}") print(f" Triggers today: {data['today_triggers']}")
print(f" 成功触发次数: {data['successful_triggers']}") print(f" Successful triggers: {data['successful_triggers']}")
return True return True
else: else:
print(f"❌ 获取统计信息失败: {response.status_code}") print(f"❌ Statistics retrieval failed: {response.status_code}")
return False return False
except Exception as e: except Exception as e:
print(f"❌ 统计信息异常: {e}") print(f"❌ Statistics exception: {e}")
return False return False
def test_logs(token): def test_logs(token):
"""测试日志功能""" """Test log functionality"""
print("\n📝 测试日志功能...") print("\n📝 Testing log functionality...")
headers = {"Authorization": f"Bearer {token}"} headers = {"Authorization": f"Bearer {token}"}
try: try:
response = requests.get(f"{BASE_URL}/api/logs", headers=headers) response = requests.get(f"{BASE_URL}/api/logs", headers=headers)
if response.status_code == 200: if response.status_code == 200:
data = response.json() data = response.json()
print(f"✅ 获取日志成功，共 {len(data['logs'])} 条记录") print(f"✅ Logs retrieved successfully, total {len(data['logs'])} records")
return True return True
else: else:
print(f"❌ 获取日志失败: {response.status_code}") print(f"❌ Log retrieval failed: {response.status_code}")
return False return False
except Exception as e: except Exception as e:
print(f"❌ 日志功能异常: {e}") print(f"❌ Log functionality exception: {e}")
return False return False
def main(): def main():
"""主测试函数""" """Main test function"""
print("🚀 Gitea Webhook Ambassador 增强版功能测试") print("🚀 Gitea Webhook Ambassador Enhanced Feature Test")
print("=" * 50) print("=" * 50)
# 测试健康检查 # Test health check
if not test_health_check(): if not test_health_check():
print("❌ 健康检查失败，服务可能未启动") print("❌ Health check failed, service may not be running")
return return
# 测试登录 # Test login
token = test_login() token = test_login()
if not token: if not token:
print("❌ 登录失败，无法继续测试") print("❌ Login failed, cannot continue testing")
return return
# 测试各项功能 # Test features
test_api_key_management(token) test_api_key_management(token)
test_project_management(token) test_project_management(token)
test_stats(token) test_stats(token)
test_logs(token) test_logs(token)
print("\n" + "=" * 50) print("\n" + "=" * 50)
print("🎉 增强版功能测试完成！") print("🎉 Enhanced feature test completed!")
print("\n📋 已实现的功能:") print("\n📋 Implemented features:")
print(" ✅ Web 登录界面") print(" ✅ Web login interface")
print(" ✅ 数据库存储 API 密钥") print(" ✅ Database storage for API keys")
print(" ✅ 延长 JWT 有效期 (7天)") print(" ✅ Extended JWT validity (7 days)")
print(" ✅ 前端仪表板") print(" ✅ Frontend dashboard")
print(" ✅ 项目管理") print(" ✅ Project management")
print(" ✅ API 密钥管理") print(" ✅ API key management")
print(" ✅ 日志查看") print(" ✅ Log viewing")
print(" ✅ 健康状态监控") print(" ✅ Health status monitoring")
print("\n🌐 访问地址:") print("\n🌐 Access URLs:")
print(f" 登录页面: {BASE_URL}/login") print(f" Login page: {BASE_URL}/login")
print(f" 仪表板: {BASE_URL}/dashboard") print(f" Dashboard: {BASE_URL}/dashboard")
print(f" 管理员密钥: {ADMIN_SECRET_KEY}")
if __name__ == "__main__": if __name__ == "__main__":
main() main()

142
apps/gitea-webhook-ambassador-python/test_webhook.py
View File

@ -1,7 +1,7 @@
#!/usr/bin/env python3 #!/usr/bin/env python3
""" """
Webhook 功能测试脚本 Webhook feature test script
用于验证 Gitea Webhook Ambassador 的各项功能 Used to verify all features of Gitea Webhook Ambassador
""" """
import asyncio import asyncio
@ -10,11 +10,11 @@ import httpx
import time import time
from datetime import datetime from datetime import datetime
# 测试配置 # Test configuration
BASE_URL = "http://localhost:8000" BASE_URL = "http://localhost:8000"
WEBHOOK_SECRET = "your-secret-key-here-make-it-long-and-random" WEBHOOK_SECRET = "your-secret-key-here-make-it-long-and-random"
# 测试数据 # Test data
TEST_WEBHOOK_DATA = { TEST_WEBHOOK_DATA = {
"ref": "refs/heads/dev", "ref": "refs/heads/dev",
"before": "abc1234567890abcdef1234567890abcdef123456", "before": "abc1234567890abcdef1234567890abcdef123456",
@ -59,46 +59,46 @@ TEST_WEBHOOK_DATA = {
async def test_health_check(): async def test_health_check():
"""测试健康检查""" """Test health check"""
print("🔍 测试健康检查...") print("🔍 Testing health check...")
async with httpx.AsyncClient() as client: async with httpx.AsyncClient() as client:
try: try:
response = await client.get(f"{BASE_URL}/health") response = await client.get(f"{BASE_URL}/health")
if response.status_code == 200: if response.status_code == 200:
data = response.json() data = response.json()
print(f"✅ 健康检查通过: {data['status']}") print(f"✅ Health check passed: {data['status']}")
return True return True
else: else:
print(f"❌ 健康检查失败: {response.status_code}") print(f"❌ Health check failed: {response.status_code}")
return False return False
except Exception as e: except Exception as e:
print(f"❌ 健康检查异常: {e}") print(f"❌ Health check exception: {e}")
return False return False
async def test_queue_status(): async def test_queue_status():
"""测试队列状态""" """Test queue status"""
print("🔍 测试队列状态...") print("🔍 Testing queue status...")
async with httpx.AsyncClient() as client: async with httpx.AsyncClient() as client:
try: try:
response = await client.get(f"{BASE_URL}/health/queue") response = await client.get(f"{BASE_URL}/health/queue")
if response.status_code == 200: if response.status_code == 200:
data = response.json() data = response.json()
print(f"✅ 队列状态: {data['queue_stats']}") print(f"✅ Queue status: {data['queue_stats']}")
return True return True
else: else:
print(f"❌ 队列状态检查失败: {response.status_code}") print(f"❌ Queue status check failed: {response.status_code}")
return False return False
except Exception as e: except Exception as e:
print(f"❌ 队列状态检查异常: {e}") print(f"❌ Queue status exception: {e}")
return False return False
async def test_webhook_endpoint(): async def test_webhook_endpoint():
"""测试 Webhook 端点""" """Test webhook endpoint"""
print("🔍 测试 Webhook 端点...") print("🔍 Testing webhook endpoint...")
headers = { headers = {
"Content-Type": "application/json", "Content-Type": "application/json",
@ -113,47 +113,47 @@ async def test_webhook_endpoint():
json=TEST_WEBHOOK_DATA json=TEST_WEBHOOK_DATA
) )
print(f"📊 响应状态: {response.status_code}") print(f"📊 Response status: {response.status_code}")
print(f"📊 响应内容: {response.text}") print(f"📊 Response content: {response.text}")
if response.status_code in [200, 202]: if response.status_code in [200, 202]:
print("✅ Webhook 端点测试通过") print("✅ Webhook endpoint test passed")
return True return True
else: else:
print(f"❌ Webhook 端点测试失败: {response.status_code}") print(f"❌ Webhook endpoint test failed: {response.status_code}")
return False return False
except Exception as e: except Exception as e:
print(f"❌ Webhook 端点测试异常: {e}") print(f"❌ Webhook endpoint exception: {e}")
return False return False
async def test_metrics_endpoint(): async def test_metrics_endpoint():
"""测试监控指标端点""" """Test metrics endpoint"""
print("🔍 测试监控指标端点...") print("🔍 Testing metrics endpoint...")
async with httpx.AsyncClient() as client: async with httpx.AsyncClient() as client:
try: try:
response = await client.get(f"{BASE_URL}/metrics") response = await client.get(f"{BASE_URL}/metrics")
if response.status_code == 200: if response.status_code == 200:
print("✅ 监控指标端点测试通过") print("✅ Metrics endpoint test passed")
# 打印一些关键指标 # Print some key metrics
content = response.text content = response.text
for line in content.split('\n'): for line in content.split('\n'):
if 'webhook_requests_total' in line or 'queue_size' in line: if 'webhook_requests_total' in line or 'queue_size' in line:
print(f"📊 {line}") print(f"📊 {line}")
return True return True
else: else:
print(f"❌ 监控指标端点测试失败: {response.status_code}") print(f"❌ Metrics endpoint test failed: {response.status_code}")
return False return False
except Exception as e: except Exception as e:
print(f"❌ 监控指标端点测试异常: {e}") print(f"❌ Metrics endpoint exception: {e}")
return False return False
async def test_deduplication(): async def test_deduplication():
"""测试防抖功能""" """Test deduplication feature"""
print("🔍 测试防抖功能...") print("🔍 Testing deduplication feature...")
headers = { headers = {
"Content-Type": "application/json", "Content-Type": "application/json",
@ -162,64 +162,64 @@ async def test_deduplication():
async with httpx.AsyncClient() as client: async with httpx.AsyncClient() as client:
try: try:
# 第一次请求 # First request
print("📤 发送第一次请求...") print("📤 Sending first request...")
response1 = await client.post( response1 = await client.post(
f"{BASE_URL}/webhook/gitea", f"{BASE_URL}/webhook/gitea",
headers=headers, headers=headers,
json=TEST_WEBHOOK_DATA json=TEST_WEBHOOK_DATA
) )
print(f"📊 第一次响应: {response1.status_code}") print(f"📊 First response: {response1.status_code}")
# 等待一秒 # Wait one second
await asyncio.sleep(1) await asyncio.sleep(1)
# 第二次请求（相同数据，应该被防抖） # Second request (same data, should be deduplicated)
print("📤 发送第二次请求（相同数据）...") print("📤 Sending second request (same data)...")
response2 = await client.post( response2 = await client.post(
f"{BASE_URL}/webhook/gitea", f"{BASE_URL}/webhook/gitea",
headers=headers, headers=headers,
json=TEST_WEBHOOK_DATA json=TEST_WEBHOOK_DATA
) )
print(f"📊 第二次响应: {response2.status_code}") print(f"📊 Second response: {response2.status_code}")
# 修改提交哈希，发送第三次请求 # Modify commit hash, send third request
modified_data = TEST_WEBHOOK_DATA.copy() modified_data = TEST_WEBHOOK_DATA.copy()
modified_data["after"] = "ghi1234567890abcdef1234567890abcdef123456" modified_data["after"] = "ghi1234567890abcdef1234567890abcdef123456"
print("📤 发送第三次请求（不同提交哈希）...") print("📤 Sending third request (different commit hash)...")
response3 = await client.post( response3 = await client.post(
f"{BASE_URL}/webhook/gitea", f"{BASE_URL}/webhook/gitea",
headers=headers, headers=headers,
json=modified_data json=modified_data
) )
print(f"📊 第三次响应: {response3.status_code}") print(f"📊 Third response: {response3.status_code}")
print("✅ 防抖功能测试完成") print("✅ Deduplication feature test completed")
return True return True
except Exception as e: except Exception as e:
print(f"❌ 防抖功能测试异常: {e}") print(f"❌ Deduplication feature exception: {e}")
return False return False
async def test_invalid_webhook(): async def test_invalid_webhook():
"""测试无效的 Webhook 请求""" """Test invalid webhook requests"""
print("🔍 测试无效的 Webhook 请求...") print("🔍 Testing invalid webhook requests...")
async with httpx.AsyncClient() as client: async with httpx.AsyncClient() as client:
try: try:
# 测试缺少签名 # Test missing signature
print("📤 测试缺少签名...") print("📤 Testing missing signature...")
response1 = await client.post( response1 = await client.post(
f"{BASE_URL}/webhook/gitea", f"{BASE_URL}/webhook/gitea",
headers={"Content-Type": "application/json"}, headers={"Content-Type": "application/json"},
json=TEST_WEBHOOK_DATA json=TEST_WEBHOOK_DATA
) )
print(f"📊 缺少签名响应: {response1.status_code}") print(f"📊 Missing signature response: {response1.status_code}")
# 测试错误的签名 # Test wrong signature
print("📤 测试错误的签名...") print("📤 Testing wrong signature...")
response2 = await client.post( response2 = await client.post(
f"{BASE_URL}/webhook/gitea", f"{BASE_URL}/webhook/gitea",
headers={ headers={
@ -228,10 +228,10 @@ async def test_invalid_webhook():
}, },
json=TEST_WEBHOOK_DATA json=TEST_WEBHOOK_DATA
) )
print(f"📊 错误签名响应: {response2.status_code}") print(f"📊 Wrong signature response: {response2.status_code}")
# 测试无效的 JSON # Test invalid JSON
print("📤 测试无效的 JSON...") print("📤 Testing invalid JSON...")
response3 = await client.post( response3 = await client.post(
f"{BASE_URL}/webhook/gitea", f"{BASE_URL}/webhook/gitea",
headers={ headers={
@ -240,28 +240,28 @@ async def test_invalid_webhook():
}, },
content="invalid json" content="invalid json"
) )
print(f"📊 无效 JSON 响应: {response3.status_code}") print(f"📊 Invalid JSON response: {response3.status_code}")
print("✅ 无效请求测试完成") print("✅ Invalid request tests completed")
return True return True
except Exception as e: except Exception as e:
print(f"❌ 无效请求测试异常: {e}") print(f"❌ Invalid request tests exception: {e}")
return False return False
async def main(): async def main():
"""主测试函数""" """Main test function"""
print("🚀 开始 Gitea Webhook Ambassador 功能测试") print("🚀 Starting Gitea Webhook Ambassador feature tests")
print("=" * 50) print("=" * 50)
tests = [ tests = [
("健康检查", test_health_check), ("Health Check", test_health_check),
("队列状态", test_queue_status), ("Queue Status", test_queue_status),
("Webhook 端点", test_webhook_endpoint), ("Webhook Endpoint", test_webhook_endpoint),
("监控指标", test_metrics_endpoint), ("Metrics", test_metrics_endpoint),
("防抖功能", test_deduplication), ("Deduplication", test_deduplication),
("无效请求", test_invalid_webhook), ("Invalid Requests", test_invalid_webhook),
] ]
results = [] results = []
@ -274,34 +274,34 @@ async def main():
result = await test_func() result = await test_func()
results.append((test_name, result)) results.append((test_name, result))
except Exception as e: except Exception as e:
print(f"❌ {test_name} 测试异常: {e}") print(f"❌ {test_name} test exception: {e}")
results.append((test_name, False)) results.append((test_name, False))
# 等待一下再进行下一个测试 # Wait a bit before next test
await asyncio.sleep(1) await asyncio.sleep(1)
# 输出测试结果 # Output test results
print("\n" + "=" * 50) print("\n" + "=" * 50)
print("📊 测试结果汇总") print("📊 Test Results Summary")
print("=" * 50) print("=" * 50)
passed = 0 passed = 0
total = len(results) total = len(results)
for test_name, result in results: for test_name, result in results:
status = "✅ 通过" if result else "❌ 失败" status = "✅ Passed" if result else "❌ Failed"
print(f"{test_name}: {status}") print(f"{test_name}: {status}")
if result: if result:
passed += 1 passed += 1
print(f"\n📈 总体结果: {passed}/{total} 测试通过") print(f"\n📈 Overall: {passed}/{total} tests passed")
if passed == total: if passed == total:
print("🎉 所有测试通过！服务运行正常。") print("🎉 All tests passed! Service is running normally.")
else: else:
print("⚠️ 部分测试失败，请检查服务配置和日志。") print("⚠️ Some tests failed, please check service configuration and logs.")
if __name__ == "__main__": if __name__ == "__main__":
# 运行测试 # Run tests
asyncio.run(main()) asyncio.run(main())