This directory contains separate, optimized Docker containers for PieFed designed specifically for Kubernetes deployment with your infrastructure.

🏗️ Architecture Overview

Multi-Container Design

piefed-base - Shared foundation image with all PieFed dependencies
piefed-web - Web server handling HTTP requests (Python/Flask + Nginx)
piefed-worker - Background job processing (Celery workers + Scheduler)
Database Init Job - One-time migration job that runs before deployments

Why Separate Containers?

✅ Independent Scaling: Scale web and workers separately based on load
✅ Better Resource Management: Optimize CPU/memory for each workload type
✅ Enhanced Monitoring: Separate metrics for web performance vs queue processing
✅ Fault Isolation: Web issues don't affect background processing and vice versa
✅ Rolling Updates: Update web and workers independently
✅ Kubernetes Native: Works perfectly with HPA, resource limits, and service mesh

🚀 Quick Start

Build All Containers

# From the build/piefed directory
./build-all.sh

This will:

Build the base image with all PieFed dependencies
Build the web container with Nginx + Python/Flask (uWSGI)
Build the worker container with Celery workers
Push to your Harbor registry: registry.keyboardvagabond.com

Individual Container Builds

# Build just web container
cd piefed-web && docker build --platform linux/arm64 \
  -t registry.keyboardvagabond.com/library/piefed-web:latest .

# Build just worker container  
cd piefed-worker && docker build --platform linux/arm64 \
  -t registry.keyboardvagabond.com/library/piefed-worker:latest .

📦 Container Details

piefed-web - Web Server Container

Purpose: Handle HTTP requests, API calls, federation endpoints
Components:

Nginx (optimized with rate limiting, gzip, security headers)
Python/Flask with uWSGI (tuned for web workload)
Static asset serving with CDN fallback

Resources: Optimized for HTTP response times
Health Check: curl -f http://localhost:80/api/health
Scaling: Based on HTTP traffic, CPU usage

piefed-worker - Background Job Container

Purpose: Process federation, image optimization, emails, scheduled tasks
Components:

Celery workers (background task processing)
Celery beat (cron-like task scheduling)
Redis for task queue management

Resources: Optimized for background processing throughput
Health Check: celery inspect ping
Scaling: Based on queue depth, memory usage

⚙️ Configuration

Environment Variables

Both containers share the same configuration:

Required

PIEFED_DOMAIN=piefed.keyboardvagabond.com
DB_HOST=postgresql-shared-rw.postgresql-system.svc.cluster.local
DB_NAME=piefed
DB_USER=piefed_user
DB_PASSWORD=secure_password_here

Redis Configuration

REDIS_HOST=redis-ha-haproxy.redis-system.svc.cluster.local
REDIS_PORT=6379
REDIS_PASSWORD=redis_password_if_needed

S3 Media Storage (Backblaze B2)

# S3 Configuration for media storage
S3_ENABLED=true
S3_BUCKET=piefed-bucket
S3_REGION=eu-central-003
S3_ENDPOINT=https://s3.eu-central-003.backblazeb2.com
S3_ACCESS_KEY=your_b2_key_id
S3_SECRET_KEY=your_b2_secret_key
S3_PUBLIC_URL=https://pfm.keyboardvagabond.com/

Email (Mailgun)

MAIL_SERVER=smtp.eu.mailgun.org
MAIL_PORT=587
MAIL_USERNAME=piefed@mail.keyboardvagabond.com
MAIL_PASSWORD=<mail password>
MAIL_USE_TLS=true
MAIL_DEFAULT_SENDER=piefed@mail.keyboardvagabond.com

Database Initialization

Database migrations are handled by a separate Kubernetes Job (piefed-db-init) that runs before the web and worker deployments. This ensures:

✅ No Race Conditions: Single job runs migrations, avoiding conflicts
✅ Proper Ordering: Flux ensures Job completes before deployments start
✅ Clean Separation: Web/worker pods focus only on their roles
✅ Easier Troubleshooting: Migration issues are isolated

The init job uses a dedicated entrypoint script (entrypoint-init.sh) that:

Waits for database and Redis to be available
Runs flask db upgrade to apply migrations
Populates the community search index
Exits cleanly, allowing deployments to proceed

🎯 Deployment Strategy

Initialization Pattern

Database Init Job (piefed-db-init):
- Runs first as a Kubernetes Job
- Applies database migrations
- Populates initial data
- Must complete successfully before deployments
Web Pods:
- Start after init job completes
- No migration logic needed
- Fast startup times
Worker Pods:
- Start after init job completes
- No migration logic needed
- Focus on background processing

Scaling Recommendations

Web Containers

Start: 2 replicas for high availability
Scale Up: When CPU > 70% or response time > 200ms
Resources: 2 CPU, 4GB RAM per pod

Worker Containers

Start: 1 replica for basic workload
Scale Up: When queue depth > 100 or processing lag > 5 minutes
Resources: 1 CPU, 2GB RAM initially

📊 Monitoring Integration

OpenObserve Dashboards

Web Container Metrics

HTTP response times
Request rates by endpoint
Django request metrics
Nginx connection metrics

Worker Container Metrics

Task processing rates
Task failure rates
Celery worker status
Queue depth metrics

Health Checks

Web: HTTP-based health check

curl -f http://localhost:80/api/health

Worker: Celery status check

celery inspect ping

🔄 Updates & Maintenance

Updating PieFed Version

Update PIEFED_VERSION in piefed-base/Dockerfile
Update VERSION in build-all.sh
Run ./build-all.sh
Deploy web containers first, then workers

Rolling Updates

# 1. Run migrations if needed (for version upgrades)
kubectl delete job piefed-db-init -n piefed-application
kubectl apply -f manifests/applications/piefed/job-db-init.yaml
kubectl wait --for=condition=complete --timeout=300s job/piefed-db-init -n piefed-application

# 2. Update web containers
kubectl rollout restart deployment piefed-web -n piefed-application
kubectl rollout status deployment piefed-web -n piefed-application

# 3. Update workers
kubectl rollout restart deployment piefed-worker -n piefed-application
kubectl rollout status deployment piefed-worker -n piefed-application

🛠️ Troubleshooting

Common Issues

Database Connection & Migrations

# Check migration status
kubectl exec -it piefed-web-xxx -- flask db current

# View migration history
kubectl exec -it piefed-web-xxx -- flask db history

# Run migrations manually (if needed)
kubectl exec -it piefed-web-xxx -- flask db upgrade

# Check Flask shell access
kubectl exec -it piefed-web-xxx -- flask shell

Queue Processing

# Check Celery status
kubectl exec -it piefed-worker-xxx -- celery inspect active

# View queue stats
kubectl exec -it piefed-worker-xxx -- celery inspect stats

Storage Issues

# Test S3 connection
kubectl exec -it piefed-web-xxx -- python manage.py check

# Check static files  
curl -v https://piefed.keyboardvagabond.com/static/css/style.css

🔗 Integration with Your Infrastructure

Perfect Fit For Your Setup

✅ PostgreSQL: Uses your CloudNativePG cluster with read replicas
✅ Redis: Integrates with your Redis cluster
✅ S3 Storage: Leverages Backblaze B2 + Cloudflare CDN
✅ Monitoring: Ready for OpenObserve metrics collection
✅ SSL: Works with your cert-manager + Let's Encrypt setup
✅ DNS: Compatible with external-dns + Cloudflare
✅ CronJobs: Kubernetes-native scheduled tasks

Next Steps

✅ Build containers with ./build-all.sh
✅ Create Kubernetes manifests for both deployments
✅ Set up PostgreSQL database and user
✅ Configure ingress for piefed.keyboardvagabond.com
✅ Set up maintenance CronJobs
✅ Configure monitoring with OpenObserve

Built with ❤️ for your sophisticated Kubernetes infrastructure