freeleaps-ops/apps/gitea-webhook-ambassador-python

Gitea Webhook Ambassador (Python)

A high-performance Python webhook service for connecting Gitea and Jenkins, supporting intelligent dispatch, high concurrency processing, and deduplication strategies.

🚀 Features

1. Intelligent Dispatch Strategy

• dev branches → Trigger alpha environment build
• prod branches → Trigger production environment build
• other branches → Configurable default strategy

2. High Concurrency Processing

• Asynchronous Task Queue: Use Celery + Redis for high concurrency
• Task Queueing Mechanism: Prevent build loss, ensure tasks are executed in order
• Load Balancing: Support multiple worker instances

3. Deduplication Strategy

• Deduplication based on commit hash + branch: Prevent repeated triggers
• Time Window Deduplication: Only trigger once for the same commit within a specified time window
• Intelligent Deduplication: Support configurable deduplication strategies

🏗️ Architecture Design

Gitea Webhook → FastAPI → Celery Queue → Jenkins Workers
     ↓              ↓           ↓            ↓
Signature Verification  Routing Dispatch  Task Queueing  Concurrent Execution
     ↓              ↓           ↓            ↓
Deduplication   Environment Judgment  Persistent Storage  Status Feedback

📁 Project Structure

gitea-webhook-ambassador-python/
├── app/
│   ├── __init__.py
│   ├── main.py                 # FastAPI application entry
│   ├── config.py               # Configuration management
│   ├── models/                 # Data models
│   │   ├── __init__.py
│   │   ├── gitea.py           # Gitea webhook model
│   │   └── jenkins.py         # Jenkins job model
│   ├── services/              # Business logic
│   │   ├── __init__.py
│   │   ├── webhook_service.py # Webhook processing service
│   │   ├── jenkins_service.py # Jenkins integration service
│   │   ├── queue_service.py   # Queue management service
│   │   └── dedup_service.py   # Deduplication service
│   ├── api/                   # API routes
│   │   ├── __init__.py
│   │   ├── webhook.py         # Webhook endpoint
│   │   ├── health.py          # Health check
│   │   └── admin.py           # Admin interface
│   ├── core/                  # Core components
│   │   ├── __init__.py
│   │   ├── security.py        # Security validation
│   │   ├── database.py        # Database connection
│   │   └── cache.py           # Cache management
│   └── tasks/                 # Celery tasks
│       ├── __init__.py
│       └── jenkins_tasks.py   # Jenkins task processing
├── tests/                     # Test files
├── docker/                    # Docker configuration
├── requirements.txt           # Python dependencies
├── docker-compose.yml         # Development environment
└── README.md

🛠️ Tech Stack

• Web Framework: FastAPI
• Task Queue: Celery + Redis
• Database: PostgreSQL (production) / SQLite (development)
• Cache: Redis
• Monitoring: Prometheus + Grafana
• Logging: Structured logging with JSON
• Testing: pytest + pytest-asyncio

🚀 Quick Start

1. Install Dependencies

pip install -r requirements.txt

2. Configure Environment

cp .env.example .env
# Edit the .env file to configure Jenkins and database connections

3. Start Service

# Start Redis
docker run -d -p 6379:6379 redis:alpine

# Start Celery worker
celery -A app.tasks worker --loglevel=info

# Start FastAPI application
uvicorn app.main:app --reload --host 0.0.0.0 --port 8000

📋 Configuration

Environment Dispatch Configuration

environments:
  dev:
    branches: ["dev", "develop", "development"]
    jenkins_job: "alpha-build"
    jenkins_url: "https://jenkins-alpha.example.com"
  prod:
    branches: ["prod", "production", "main", "master"]
    jenkins_job: "production-build"
    jenkins_url: "https://jenkins-prod.example.com"
  default:
    jenkins_job: "default-build"
    jenkins_url: "https://jenkins-default.example.com"

Deduplication Configuration

deduplication:
  enabled: true
  window_seconds: 300  # 5-minute deduplication window
  strategy: "commit_branch"  # commit_hash + branch
  cache_ttl: 3600  # Cache for 1 hour

Queue Configuration

queue:
  max_concurrent: 10
  max_retries: 3
  retry_delay: 60  # seconds
  priority_levels: 3

🔧 API Endpoints

Webhook Endpoint

POST /webhook/gitea

Health Check

GET /health
GET /health/queue
GET /health/jenkins

Admin Endpoints

GET /admin/queue/status
GET /admin/queue/stats
POST /admin/queue/clear

🧪 Testing

# Run all tests
pytest

# Run specific test
pytest tests/test_webhook_service.py

# Run performance test
pytest tests/test_performance.py

📊 Monitoring Metrics

• Webhook receive rate
• Task queue length
• Jenkins build success rate
• Response time distribution
• Deduplication hit rate

🔒 Security Features

• Webhook signature verification
• API key authentication
• Request rate limiting
• Input validation and sanitization
• Secure logging