Keybard-Vagabond-Demo/manifests/applications/mastodon/README.md

# Mastodon Application

This directory contains the Mastodon fediverse application deployment for the Keyboard Vagabond cluster.

## Overview

Mastodon is a free, open-source decentralized social media platform deployed using the official Helm chart via FluxCD GitOps.

**Deployment Status**: ✅ **Phase 1 - Core Deployment** (without Elasticsearch)

- **URL**: `https://mastodon.keyboardvagabond.com`
- **Federation Domain**: `keyboardvagabond.com` (CRITICAL: Never change this!)
- **Architecture**: Multi-container design with Web, Sidekiq, and Streaming deployments
- **Authentication**: Authentik OIDC integration + local accounts
- **Storage**: Backblaze B2 S3-compatible storage with Cloudflare CDN
- **Database**: Shared PostgreSQL cluster with CloudNativePG
- **Cache**: Shared Redis cluster

## Directory Structure

```
mastodon/
├── namespace.yaml           # mastodon-application namespace
├── repository.yaml          # Official Mastodon Helm chart repository
├── secret.yaml              # SOPS-encrypted secrets (credentials, tokens)
├── helm-release.yaml        # Main HelmRelease configuration
├── ingress.yaml            # NGINX ingress with SSL and external-dns
├── monitoring.yaml         # ServiceMonitor for OpenObserve integration
├── kustomization.yaml      # Resource list
└── README.md               # This documentation
```

## 🔑 Pre-Deployment Setup

### 1. Generate Mastodon Secrets

**Important**: Replace placeholder values in `secret.yaml` before deployment:

```bash
# Generate SECRET_KEY_BASE (using modern Rails command)
docker run --rm -it tootsuite/mastodon bundle exec rails secret

# Generate OTP_SECRET (using modern Rails command)
docker run --rm -it tootsuite/mastodon bundle exec rails secret

# Generate VAPID Keys (after setting SECRET_KEY_BASE and OTP_SECRET)
docker run --rm -it \
  -e SECRET_KEY_BASE="your_secret_key_base" \
  -e OTP_SECRET="your_otp_secret" \
  tootsuite/mastodon bundle exec rake mastodon:webpush:generate_vapid_key
```

### 2. Database Setup

Create Mastodon database and user in the existing PostgreSQL cluster:

```bash
kubectl exec -it postgresql-shared-1 -n postgresql-system -- psql -U postgres
```

```sql
-- Create database and user
CREATE DATABASE mastodon_production;
CREATE USER mastodon_user WITH PASSWORD 'SECURE_PASSWORD_HERE';
GRANT ALL PRIVILEGES ON DATABASE mastodon_production TO mastodon_user;
ALTER DATABASE mastodon_production OWNER TO mastodon_user;
\q
```

### 3. Update Secret Values

Edit `secret.yaml` and replace:
- `REPLACE_WITH_GENERATED_SECRET_KEY_BASE`
- `REPLACE_WITH_GENERATED_OTP_SECRET`  
- `REPLACE_WITH_GENERATED_VAPID_PRIVATE_KEY`
- `REPLACE_WITH_GENERATED_VAPID_PUBLIC_KEY`
- `REPLACE_WITH_POSTGRESQL_PASSWORD`
- `REPLACE_WITH_REDIS_PASSWORD`

### 4. Encrypt Secrets

```bash
sops --encrypt --in-place manifests/applications/mastodon/secret.yaml
```

## 🚀 Deployment

### Add to Applications Kustomization

Add mastodon to `manifests/applications/kustomization.yaml`:

```yaml
resources:
# ... existing apps
- mastodon/
```

### Commit and Deploy

```bash
git add manifests/applications/mastodon/
git commit -m "feat: Add Mastodon fediverse application"
git push origin k8s-fleet
```

Flux will automatically deploy within 5-10 minutes.

## 📋 Post-Deployment Configuration

### 1. Initial Admin Setup

Wait for pods to be ready, then create admin account:

```bash
# Check deployment status
kubectl get pods -n mastodon-application

# Create admin account (single-user mode enabled initially)
kubectl exec -n mastodon-application deployment/mastodon-web -- \
  tootctl accounts create admin \
  --email admin@keyboardvagabond.com \
  --confirmed \
  --role Admin
```

### 2. Disable Single-User Mode

After creating admin account, edit `helm-release.yaml`:

```yaml
mastodon:
  single_user_mode: false  # Change from true to false
```

Commit and push to apply changes.

### 3. Federation Testing

Test federation with other Mastodon instances:
1. Search for accounts from other instances
2. Follow accounts from other instances  
3. Verify media attachments display correctly via CDN

## 🔧 Configuration Details

### Resource Allocation

**Starting Resources** (Phase 1):
- **Web**: 2 replicas, 1-2 CPU, 2-4Gi memory
- **Sidekiq**: 2 replicas, 0.5-1 CPU, 1-2Gi memory  
- **Streaming**: 2 replicas, 0.25-0.5 CPU, 0.5-1Gi memory
- **Total**: ~5.5 CPU requests, ~9Gi memory requests

### External Dependencies

- ✅ **PostgreSQL**: `postgresql-shared-rw.postgresql-system.svc.cluster.local:5432`
- ✅ **Redis**: `redis-ha-haproxy.redis-system.svc.cluster.local:6379`
- ✅ **S3 Storage**: Backblaze B2 `mastodon-bucket`
- ✅ **CDN**: Cloudflare `mm.keyboardvagabond.com`
- ✅ **SMTP**: `<YOUR_SMTP_SERVER>` `<YOUR_EMAIL_ADDRESS>`
- ✅ **OIDC**: Authentik `auth.keyboardvagabond.com`
- ❌ **Elasticsearch**: Not configured (Phase 2)

### Security Features

- **HTTPS**: Enforced with Let's Encrypt certificates
- **Headers**: Security headers via NGINX ingress
- **OIDC**: Single Sign-On with Authentik
- **S3**: Media storage with CDN distribution
- **Secrets**: SOPS-encrypted in Git

## 📊 Monitoring

### OpenObserve Integration

Metrics automatically collected via ServiceMonitor:
- **URL**: `https://obs.keyboardvagabond.com`
- **Metrics**: Mastodon application metrics, HTTP requests, response times
- **Logs**: Application logs via OpenTelemetry collector

### Health Checks

```bash
# Check pod status
kubectl get pods -n mastodon-application

# Check ingress and certificates
kubectl get ingress,certificates -n mastodon-application

# Check logs
kubectl logs -n mastodon-application deployment/mastodon-web
kubectl logs -n mastodon-application deployment/mastodon-sidekiq
```

## 🔄 Phase 2: Elasticsearch Integration

### When to Add Elasticsearch

Add Elasticsearch when you need:
- Full-text search within Mastodon
- Better search performance for content discovery
- Enhanced user experience with search features

### Implementation Steps

1. **Add Elasticsearch infrastructure** to `manifests/infrastructure/elasticsearch/`
2. **Uncomment Elasticsearch configuration** in `helm-release.yaml`
3. **Update dependencies** to include Elasticsearch
4. **Enable search features** in Mastodon admin panel

## 🆘 Troubleshooting

### Common Issues

**Database Connection Errors**:
```bash
# Check PostgreSQL connectivity
kubectl exec -n mastodon-application deployment/mastodon-web -- \
  pg_isready -h postgresql-shared-rw.postgresql-system.svc.cluster.local -p 5432
```

**Redis Connection Errors**:
```bash
# Check Redis connectivity  
kubectl exec -n mastodon-application deployment/mastodon-web -- \
  redis-cli -h redis-ha-haproxy.redis-system.svc.cluster.local -p 6379 ping
```

**S3 Upload Issues**:
- Verify Backblaze B2 credentials
- Check bucket permissions and CORS configuration
- Test CDN connectivity to `mm.keyboardvagabond.com`

**OIDC Authentication Issues**:
- Verify Authentik provider configuration
- Check client ID and secret
- Confirm issuer URL accessibility

### Support Commands

```bash
# Run Mastodon CLI commands
kubectl exec -n mastodon-application deployment/mastodon-web -- tootctl help

# Database migrations
kubectl exec -n mastodon-application deployment/mastodon-web -- \
  rails db:migrate

# Clear cache
kubectl exec -n mastodon-application deployment/mastodon-web -- \
  tootctl cache clear
```

## 📚 References

- **Official Documentation**: https://docs.joinmastodon.org/
- **Helm Chart**: https://github.com/mastodon/chart
- **Admin Guide**: https://docs.joinmastodon.org/admin/
- **Federation Guide**: https://docs.joinmastodon.org/spec/activitypub/