---
description: Development patterns, operational guidelines, and troubleshooting
globs: ["build/**/*", "tools/**/*", "justfile", "*.md"]
alwaysApply: false
---

# Development Patterns & Operational Guidelines

## Configuration Management
- **Kustomize**: Used for resource composition and patching via `patches/` directory
- **Helm**: Complex applications deployed via HelmRelease CRDs
- **GitOps**: All applications deployed via Flux from Git repository (`k8s-fleet` branch)
- **Staging**: Use separate branches/overlays for staging vs production environments

## Application Deployment Standards
- **Container Registry**: Use Harbor (`<YOUR_REGISTRY_URL>`) for all custom images
- **Multi-Stage Builds**: Implement for Python/Node.js applications to reduce image size by ~75%
- **Storage**: Use Longhorn with 2-replica redundancy, label volumes for S3 backup selection  
- **Database**: Leverage shared PostgreSQL cluster with dedicated databases per application
- **Monitoring**: Implement ServiceMonitor for OpenObserve integration

## Email Templates & User Onboarding
- **Community Signup**: Professional welcome email template at `docs/email-templates/community-signup.html`
- **Authentik Integration**: Uses `{AUTHENTIK_URL}` placeholder for account activation links
- **Documentation**: Complete setup guide in `docs/email-templates/README.md`
- **Services Overview**: Template showcases all fediverse services with direct links
- **Branding**: Features horizontal Keyboard Vagabond logo from Picsur CDN
- **Rate Limiting**: Implement ActivityPub inbox burst protection for all fediverse applications

## Container Build Patterns

### Multi-Stage Docker Strategy ✅ WORKING
**Key Lessons Learned**:
- **Framework Identification**: Critical to identify Flask vs Django early (different command structures)
- **Python Virtual Environment**: uWSGI must use same Python version as venv
- **Static File Paths**: Flask apps with application factory have nested structure (`/app/app/static/`)
- **Database Initialization**: Flask requires explicit `flask init-db` command
- **Log File Permissions**: Non-root users need explicit ownership of log files

### Build Process
```bash
# Build all containers
./build-all.sh

# Build specific application
cd build/app-name
docker build -t <YOUR_REGISTRY_URL>/library/app-name:tag .
docker push <YOUR_REGISTRY_URL>/library/app-name:tag
```

## Key Framework Patterns

### Flask Applications (PieFed)
- **Environment Variables**: URL-based configuration (DATABASE_URL, REDIS_URL)
- **uWSGI Integration**: Install via pip in venv, not Alpine packages
- **Static Files**: Careful nginx configuration for nested structure
- **Multi-stage Builds**: Essential to remove build dependencies

### Django Applications (BookWyrm)
- **S3 Static Files**: Theme compilation before static collection
- **Celery Beat**: Single instance only (prevents duplicate scheduling)
- **ACL Configuration**: Backblaze B2 requires empty `AWS_DEFAULT_ACL`

### Laravel Applications (Pixelfed) 
- **S3 Default Disk**: `DANGEROUSLY_SET_FILESYSTEM_DRIVER=s3` required
- **Cache Invalidation**: `php artisan config:cache` after S3 changes
- **Dedicated Buckets**: Avoid prefix conflicts with dedicated bucket approach

## Operational Tools & Management

### Administrative Access ✅ SECURED
- **kubectl Context**: `admin@keyboardvagabond-tailscale` (internal VLAN IP)
- **Tailscale Client**: CGNAT range 100.64.0.0/10 access only
- **Harbor Registry**: Direct HTTPS access (Zero Trust incompatible)

### Essential Commands
```bash
# Talos cluster management (Tailscale VPN required)
talosctl config endpoint 10.132.0.10 10.132.0.20 10.132.0.30
talosctl health

# Kubernetes cluster access
kubectl config use-context admin@keyboardvagabond-tailscale
kubectl get nodes

# SOPS secret management
sops -e -i secrets.yaml
sops -d secrets.yaml | kubectl apply -f -

# Flux GitOps management  
flux get sources all
flux reconcile source git flux-system
```

### Terminal Environment Notes
- **PowerShell on macOS**: PSReadLine may display errors but commands execute successfully
- **Terminal Preference**: Use default OS terminal over PowerShell (except Windows)
- **Command Output**: Despite display issues, outputs remain readable and functional

## Scaling Preparation
- **Node Addition**: NetCup Cloud vLAN 1004963 with sequential IPs (10.132.0.x/24)
- **Storage Scaling**: Longhorn distributed across nodes with S3 backup integration
- **Load Balancing**: MetalLB or cloud load balancer integration ready
- **High Availability**: Additional control plane nodes can be added

## Troubleshooting Patterns

### Zero Trust Issues
- **Corporate VPN Blocking**: SSL handshake failures - test from different networks
- **Service Discovery**: Check label mismatch between service selector and pod labels
- **StatefulSet Issues**: Use manual Helm deployment for immutable field changes

### Common Application Issues
- **PHP Applications**: Clear Laravel config cache after environment changes
- **Flask Applications**: Verify uWSGI Python version matches venv
- **Django Applications**: Ensure theme compilation before static file collection
- **Container Builds**: Multi-stage builds reduce size but require careful dependency management

### Network & Storage Issues
- **Longhorn**: Check replica distribution across nodes
- **S3 Backup**: Verify volume labels for backup inclusion
- **Database**: Use read replicas for read-heavy operations
- **CDN**: Dedicated buckets eliminate prefix conflicts

## Performance Optimizations
- **CDN Caching**: Cloudflare cache rules for static assets (1 year cache)
- **Image Processing**: Background workers handle optimization and federation
- **Database Optimization**: Read replicas and proper indexing
- **ActivityPub Rate Limiting**: 10r/s with 300 request burst buffer

## Future Development Guidelines
- **New Services**: Zero Trust ingress pattern mandatory (no cert-manager/external-dns)
- **Security**: Never expose external ingress ports - all traffic via Cloudflare tunnels
- **CDN Strategy**: Use dedicated S3 buckets per application
- **Subdomains**: Cloudflare Free plan supports only one level (`app.domain.com`)

@development-workflow-template.yaml
@container-build-template.dockerfile
@troubleshooting-history.mdc
@talos-config-template.yaml