Kami Spider

A stateless, production-ready FastAPI-based web service platform that hosts multiple independent web applications through a unified entry point. Built with modern Python ecosystem practices, full observability, and production deployment support.

🚀 Features

• Stateless Service Architecture: Application state externalized to Redis and MySQL
• Multi-Application Support: Single web service hosting independent applications (App A, App B, etc.)
• Modern Python Stack: Python 3.13, UV package manager, FastAPI, SQLModel, Pydantic 2.x
• Full Observability: OpenTelemetry integration with trace context propagation
• Production Deployment: Docker, Docker Compose, Docker Swarm, and Kubernetes support
• API Quality: Pydantic-validated I/O, auto-generated documentation, RESTful response format
• Comprehensive Middleware: TraceID injection, request logging, exception handling, CORS
• Structured Logging: JSON logs with trace context correlation
• Health Checks: Component-level health monitoring

📋 Technology Stack

Component	Technology	Version
Language	Python	3.13
Package Manager	UV	Latest
Web Framework	FastAPI	Latest
ORM	SQLModel	Latest
Validation	Pydantic	2.x
ASGI Server	Uvicorn	Latest
Process Manager	Gunicorn	Latest
Database	MySQL	8.x
Cache/Session	Redis	Latest
Observability	OpenTelemetry	Latest

🏗️ Architecture

System Architecture

┌─────────────────────────────────────────────────┐
│         Load Balancer / Ingress                 │
└─────────────────┬───────────────────────────────┘
                  │
        ┌─────────┴─────────┐
        │                   │
┌───────▼────────┐  ┌──────▼─────────┐
│ FastAPI Pod 1  │  │ FastAPI Pod N  │
│   (Stateless)  │  │   (Stateless)  │
└───────┬────────┘  └──────┬─────────┘
        │                   │
        └─────────┬─────────┘
                  │
        ┌─────────┴─────────┐
        │                   │
┌───────▼────────┐  ┌──────▼─────────┐
│ MySQL (Single) │  │ Redis (Single) │
│   Instance     │  │   Instance     │
└────────────────┘  └────────────────┘

Multi-Application Routing

/app-a/*  → App A (User Management)
/app-b/*  → App B (Product Management)
/health   → Health Check
/docs     → API Documentation

🛠️ Setup

Prerequisites

• Python 3.13+
• UV package manager
• MySQL 8.x (or use Docker)
• Redis (or use Docker)

Installation

1. Clone the repository

git clone <repository-url>
cd kami_spider

2. Install UV (if not already installed)

curl -LsSf https://astral.sh/uv/install.sh | sh

3. Install dependencies

uv sync

4. Configure environment

cp .env.example .env
# Edit .env with your configuration

5. Run with Docker Compose (Recommended for local development)

cd deployments/docker
docker-compose up -d

The application will be available at http://localhost:8000

Manual Setup (Without Docker)

1. Start MySQL and Redis

Ensure MySQL and Redis are running locally.

2. Update .env file

DB_HOST=localhost
DB_PORT=3306
DB_NAME=kami_spider
DB_USER=kami_user
DB_PASSWORD=kami_pass

REDIS_HOST=localhost
REDIS_PORT=6379

3. Run the application

# Development mode with auto-reload
uv run python main.py

# Or with Uvicorn directly
uv run uvicorn main:app --reload --host 0.0.0.0 --port 8000

# Production mode with Gunicorn
uv run gunicorn main:app \
  --workers 4 \
  --worker-class uvicorn.workers.UvicornWorker \
  --bind 0.0.0.0:8000

📖 API Documentation

Once running, visit:

• Swagger UI: http://localhost:8000/docs
• ReDoc: http://localhost:8000/redoc
• OpenAPI JSON: http://localhost:8000/openapi.json

API Examples

App A - User Management

# Create a user
curl -X POST http://localhost:8000/app-a/users \
  -H "Content-Type: application/json" \
  -d '{
    "username": "john_doe",
    "email": "john@example.com",
    "password": "secret123",
    "full_name": "John Doe"
  }'

# Get user
curl http://localhost:8000/app-a/users/1

# List users (paginated)
curl http://localhost:8000/app-a/users?page=1&page_size=10

App B - Product Management

# Create a product
curl -X POST http://localhost:8000/app-b/products \
  -H "Content-Type: application/json" \
  -d '{
    "name": "Laptop",
    "description": "High-performance laptop",
    "price": "999.99",
    "stock": 10,
    "sku": "LAP-001"
  }'

# Get product
curl http://localhost:8000/app-b/products/1

# List products
curl http://localhost:8000/app-b/products?page=1&page_size=10

🐳 Deployment

Docker Compose

cd deployments/docker
docker-compose up -d

Docker Swarm

# Initialize swarm
docker swarm init

# Deploy stack
docker stack deploy -c deployments/swarm/docker-compose.swarm.yml kami_spider

Kubernetes

# Apply configurations
kubectl apply -f deployments/k8s/configmap.yaml
kubectl apply -f deployments/k8s/secret.yaml
kubectl apply -f deployments/k8s/deployment.yaml
kubectl apply -f deployments/k8s/service.yaml
kubectl apply -f deployments/k8s/ingress.yaml
kubectl apply -f deployments/k8s/hpa.yaml

# Check status
kubectl get pods -l app=kami-spider
kubectl get svc kami-spider-app

🔍 Observability

OpenTelemetry

The application exports traces to the configured OpenTelemetry collector:

• Endpoint: 38.38.251.113:31547 (gRPC)
• Protocol: gRPC (insecure)
• Sampling: Configurable rate (default 100%)

Logging

Structured JSON logs in production, human-readable in development:

{
  "timestamp": "2024-01-15T10:30:00Z",
  "level": "INFO",
  "logger": "apps.app_a.services",
  "message": "User created successfully",
  "trace_id": "3fa85f64-5717-4562-b3fc-2c963f66afa6",
  "module": "services",
  "function": "create_user"
}

Health Check

curl http://localhost:8000/health

Response:

{
  "status": "healthy",
  "components": {
    "api": "healthy",
    "database": "healthy",
    "redis": "healthy"
  },
  "environment": "development",
  "version": "0.1.0"
}

📁 Project Structure

kami_spider/
├── apps/                          # Independent web applications
│   ├── app_a/                     # User management app
│   │   ├── router.py
│   │   ├── services.py
│   │   ├── models.py
│   │   └── schemas.py
│   └── app_b/                     # Product management app
│       └── ...
├── core/                          # Core infrastructure
│   ├── config.py                  # Configuration management
│   ├── database.py                # Database setup
│   ├── redis.py                   # Redis client
│   ├── responses.py               # API response formats
│   └── exceptions.py              # Custom exceptions
├── middleware/                    # Middleware components
│   ├── trace_context.py           # TraceID injection
│   ├── logging.py                 # Request/response logging
│   └── error_handler.py           # Exception handling
├── observability/                 # OpenTelemetry integration
│   ├── tracing.py                 # Distributed tracing
│   └── logging.py                 # Structured logging
├── deployments/                   # Deployment configs
│   ├── docker/
│   ├── swarm/
│   └── k8s/
├── main.py                        # Application entry point
├── pyproject.toml                 # UV project config
└── README.md

🔧 Configuration

All configuration is managed through environment variables. See .env.example for all available options.

Key Configuration Categories

• Application: APP_NAME, ENVIRONMENT, DEBUG, LOG_LEVEL
• Database: DB_HOST, DB_PORT, DB_NAME, DB_USER, DB_PASSWORD
• Redis: REDIS_HOST, REDIS_PORT, REDIS_DB
• OpenTelemetry: OTEL_ENABLED, OTEL_SERVICE_NAME, OTEL_EXPORTER_ENDPOINT
• CORS: CORS_ENABLED, CORS_ALLOW_ORIGINS

🧪 Testing

# Run tests
uv run pytest

# Run with coverage
uv run pytest --cov=. --cov-report=html

# Run specific test
uv run pytest tests/unit/test_users.py

📝 API Response Format

All API responses follow a unified structure:

{
  "code": 0,
  "message": "Success",
  "data": { ... },
  "trace_id": "3fa85f64-5717-4562-b3fc-2c963f66afa6",
  "timestamp": "2024-01-15T10:30:00Z"
}

Business Codes

Code	Description
0	Success
1001-1999	Authentication & Authorization
2000-2999	Business Logic Errors
3000-3999	Validation Errors
4000-4999	Resource Errors
5000-5999	System Errors
9000-9999	Unknown Errors

🤝 Contributing

1. Fork the repository
2. Create your feature branch (git checkout -b feature/amazing-feature)
3. Commit your changes (git commit -m 'Add amazing feature')
4. Push to the branch (git push origin feature/amazing-feature)
5. Open a Pull Request

📄 License

This project is licensed under the MIT License.

🙏 Acknowledgments

• FastAPI for the excellent web framework
• UV for modern Python package management
• OpenTelemetry for observability standards
• SQLModel for elegant database ORM

📞 Support

For issues and questions:

• Open an issue on GitHub
• Check the API documentation at /docs
• Review logs with trace IDs for debugging