Keybard-Vagabond-Demo/manifests/applications/mastodon

Mastodon Application

This directory contains the Mastodon fediverse application deployment for the Keyboard Vagabond cluster.

Overview

Mastodon is a free, open-source decentralized social media platform deployed using the official Helm chart via FluxCD GitOps.

Deployment Status: ✅ Phase 1 - Core Deployment (without Elasticsearch)

• URL: https://mastodon.keyboardvagabond.com
• Federation Domain: keyboardvagabond.com (CRITICAL: Never change this!)
• Architecture: Multi-container design with Web, Sidekiq, and Streaming deployments
• Authentication: Authentik OIDC integration + local accounts
• Storage: Backblaze B2 S3-compatible storage with Cloudflare CDN
• Database: Shared PostgreSQL cluster with CloudNativePG
• Cache: Shared Redis cluster

Directory Structure

mastodon/
├── namespace.yaml           # mastodon-application namespace
├── repository.yaml          # Official Mastodon Helm chart repository
├── secret.yaml              # SOPS-encrypted secrets (credentials, tokens)
├── helm-release.yaml        # Main HelmRelease configuration
├── ingress.yaml            # NGINX ingress with SSL and external-dns
├── monitoring.yaml         # ServiceMonitor for OpenObserve integration
├── kustomization.yaml      # Resource list
└── README.md               # This documentation

🔑 Pre-Deployment Setup

1. Generate Mastodon Secrets

Important: Replace placeholder values in secret.yaml before deployment:

# Generate SECRET_KEY_BASE (using modern Rails command)
docker run --rm -it tootsuite/mastodon bundle exec rails secret

# Generate OTP_SECRET (using modern Rails command)
docker run --rm -it tootsuite/mastodon bundle exec rails secret

# Generate VAPID Keys (after setting SECRET_KEY_BASE and OTP_SECRET)
docker run --rm -it \
  -e SECRET_KEY_BASE="your_secret_key_base" \
  -e OTP_SECRET="your_otp_secret" \
  tootsuite/mastodon bundle exec rake mastodon:webpush:generate_vapid_key

2. Database Setup

Create Mastodon database and user in the existing PostgreSQL cluster:

kubectl exec -it postgresql-shared-1 -n postgresql-system -- psql -U postgres

-- Create database and user
CREATE DATABASE mastodon_production;
CREATE USER mastodon_user WITH PASSWORD 'SECURE_PASSWORD_HERE';
GRANT ALL PRIVILEGES ON DATABASE mastodon_production TO mastodon_user;
ALTER DATABASE mastodon_production OWNER TO mastodon_user;
\q

3. Update Secret Values

Edit secret.yaml and replace:

• REPLACE_WITH_GENERATED_SECRET_KEY_BASE
• REPLACE_WITH_GENERATED_OTP_SECRET
• REPLACE_WITH_GENERATED_VAPID_PRIVATE_KEY
• REPLACE_WITH_GENERATED_VAPID_PUBLIC_KEY
• REPLACE_WITH_POSTGRESQL_PASSWORD
• REPLACE_WITH_REDIS_PASSWORD

4. Encrypt Secrets

sops --encrypt --in-place manifests/applications/mastodon/secret.yaml

🚀 Deployment

Add to Applications Kustomization

Add mastodon to manifests/applications/kustomization.yaml:

resources:
# ... existing apps
- mastodon/

Commit and Deploy

git add manifests/applications/mastodon/
git commit -m "feat: Add Mastodon fediverse application"
git push origin k8s-fleet

Flux will automatically deploy within 5-10 minutes.

📋 Post-Deployment Configuration

1. Initial Admin Setup

Wait for pods to be ready, then create admin account:

# Check deployment status
kubectl get pods -n mastodon-application

# Create admin account (single-user mode enabled initially)
kubectl exec -n mastodon-application deployment/mastodon-web -- \
  tootctl accounts create admin \
  --email admin@keyboardvagabond.com \
  --confirmed \
  --role Admin

2. Disable Single-User Mode

After creating admin account, edit helm-release.yaml:

mastodon:
  single_user_mode: false  # Change from true to false

Commit and push to apply changes.

3. Federation Testing

Test federation with other Mastodon instances:

1. Search for accounts from other instances
2. Follow accounts from other instances
3. Verify media attachments display correctly via CDN

🔧 Configuration Details

Resource Allocation

Starting Resources (Phase 1):

• Web: 2 replicas, 1-2 CPU, 2-4Gi memory
• Sidekiq: 2 replicas, 0.5-1 CPU, 1-2Gi memory
• Streaming: 2 replicas, 0.25-0.5 CPU, 0.5-1Gi memory
• Total: ~5.5 CPU requests, ~9Gi memory requests

External Dependencies

• ✅ PostgreSQL: postgresql-shared-rw.postgresql-system.svc.cluster.local:5432
• ✅ Redis: redis-ha-haproxy.redis-system.svc.cluster.local:6379
• ✅ S3 Storage: Backblaze B2 mastodon-bucket
• ✅ CDN: Cloudflare mm.keyboardvagabond.com
• ✅ SMTP: <YOUR_SMTP_SERVER> <YOUR_EMAIL_ADDRESS>
• ✅ OIDC: Authentik auth.keyboardvagabond.com
• ❌ Elasticsearch: Not configured (Phase 2)

Security Features

• HTTPS: Enforced with Let's Encrypt certificates
• Headers: Security headers via NGINX ingress
• OIDC: Single Sign-On with Authentik
• S3: Media storage with CDN distribution
• Secrets: SOPS-encrypted in Git

📊 Monitoring

OpenObserve Integration

Metrics automatically collected via ServiceMonitor:

• URL: https://obs.keyboardvagabond.com
• Metrics: Mastodon application metrics, HTTP requests, response times
• Logs: Application logs via OpenTelemetry collector

Health Checks

# Check pod status
kubectl get pods -n mastodon-application

# Check ingress and certificates
kubectl get ingress,certificates -n mastodon-application

# Check logs
kubectl logs -n mastodon-application deployment/mastodon-web
kubectl logs -n mastodon-application deployment/mastodon-sidekiq

🔄 Phase 2: Elasticsearch Integration

When to Add Elasticsearch

Add Elasticsearch when you need:

• Full-text search within Mastodon
• Better search performance for content discovery
• Enhanced user experience with search features

Implementation Steps

1. Add Elasticsearch infrastructure to manifests/infrastructure/elasticsearch/
2. Uncomment Elasticsearch configuration in helm-release.yaml
3. Update dependencies to include Elasticsearch
4. Enable search features in Mastodon admin panel

🆘 Troubleshooting

Common Issues

Database Connection Errors:

# Check PostgreSQL connectivity
kubectl exec -n mastodon-application deployment/mastodon-web -- \
  pg_isready -h postgresql-shared-rw.postgresql-system.svc.cluster.local -p 5432

Redis Connection Errors:

# Check Redis connectivity  
kubectl exec -n mastodon-application deployment/mastodon-web -- \
  redis-cli -h redis-ha-haproxy.redis-system.svc.cluster.local -p 6379 ping

S3 Upload Issues:

• Verify Backblaze B2 credentials
• Check bucket permissions and CORS configuration
• Test CDN connectivity to mm.keyboardvagabond.com

OIDC Authentication Issues:

• Verify Authentik provider configuration
• Check client ID and secret
• Confirm issuer URL accessibility

Support Commands

# Run Mastodon CLI commands
kubectl exec -n mastodon-application deployment/mastodon-web -- tootctl help

# Database migrations
kubectl exec -n mastodon-application deployment/mastodon-web -- \
  rails db:migrate

# Clear cache
kubectl exec -n mastodon-application deployment/mastodon-web -- \
  tootctl cache clear

📚 References

• Official Documentation: https://docs.joinmastodon.org/
• Helm Chart: https://github.com/mastodon/chart
• Admin Guide: https://docs.joinmastodon.org/admin/
• Federation Guide: https://docs.joinmastodon.org/spec/activitypub/