redaction (#1)

Add the redacted source file for demo purposes Reviewed-on: https://source.michaeldileo.org/michael_dileo/Keybard-Vagabond-Demo/pulls/1 Co-authored-by: Michael DiLeo <michael_dileo@proton.me> Co-committed-by: Michael DiLeo <michael_dileo@proton.me>
2025-12-24 13:40:47 +00:00
parent 612235d52b
commit 7327d77dcd
333 changed files with 39286 additions and 1 deletions
--- a/docs/ZeroTrustMigration.md
+++ b/docs/ZeroTrustMigration.md
@@ -0,0 +1,265 @@
+# Migrating from External DNS to CF Zero Trust
+Now that the CF domain is set up, it's time to move other apps and services to using it, then to potentially seal off
+as much of the Talos and k8s ports as I can.
+
+## Zero-Downtime Migration Process
+
+### Step 1: Discover Service Configuration
+```bash
+# Find service name and port
+kubectl get svc -n <namespace>
+# Example output: service-name ClusterIP 10.x.x.x <none> 9898/TCP
+```
+
+### Step 2: Create Tunnel Route (FIRST!)
+1. Go to **Cloudflare Zero Trust Dashboard** → **Networks** → **Tunnels**
+2. Find your tunnel, click **Configure**
+3. Add **Public Hostname**:
+   - **Subdomain**: `app`
+   - **Domain**: `keyboardvagabond.com`
+   - **Service**: `http://service-name.namespace.svc.cluster.local:port`
+4. **Test** the tunnel URL works before proceeding!
+
+### Step 3: Update Application Configuration
+Clear external-DNS annotations and TLS configuration:
+```yaml
+# In Helm values or ingress manifest:
+ingress:
+  annotations: {}  # Explicitly empty - removes cert-manager and external-dns
+  tls: []          # Explicitly empty array - no certificates needed
+```
+
+### Step 4: Deploy Changes
+```bash
+# For Helm apps via Flux:
+flux reconcile helmrelease <app-name> -n <namespace>
+
+# For direct manifests:
+kubectl apply -f <manifest-file>
+```
+
+### Step 5: Clean Up Certificates
+```bash
+# Delete certificate resources
+kubectl delete certificate <cert-name> -n <namespace>
+
+# Find and delete TLS secrets
+kubectl get secrets -n <namespace> | grep tls
+kubectl delete secret <tls-secret-name> -n <namespace>
+```
+
+### Step 6: Verify Clean State
+```bash
+# Check no new certificates are being created
+kubectl get certificate,secret -n <namespace> | grep <app-name>
+
+# Should only show Helm release secrets, no certificate or TLS secrets
+```
+
+### Step 7: DNS Record Management
+**How it works:**
+- **Tunnel automatically creates**: CNAME record → `tunnel-id.cfargotunnel.com`
+- **External-DNS created**: A records → your cluster IPs
+- **DNS Priority**: CNAME takes precedence over A records
+
+**Cleanup options:**
+```bash
+# Option 1: Auto-cleanup (recommended) - wait 5 minutes after removing annotations
+# External-DNS will automatically delete A records after TTL expires
+
+# Option 2: Manual cleanup (immediate)
+# Go to Cloudflare DNS dashboard and manually delete A records
+# Keep the CNAME record (created by tunnel)
+```
+
+**Verification:**
+```bash
+# Check DNS resolution shows CNAME (not A records)
+dig podinfo.keyboardvagabond.com
+
+# Should show:
+# podinfo.keyboardvagabond.com. CNAME tunnel-id.cfargotunnel.com.
+```
+
+## Rollback Plan
+If tunnel doesn't work:
+1. **Revert** Helm values/manifests (add back annotations and TLS)
+2. **Redeploy**: `flux reconcile` or `kubectl apply`
+3. **Wait** for cert-manager to recreate certificates
+
+## Benefits After Migration
+- ✅ **No exposed public IPs** - cluster nodes not directly accessible
+- ✅ **Automatic DDoS protection** via Cloudflare
+- ✅ **Centralized SSL management** - Cloudflare handles certificates
+- ✅ **Better observability** - Cloudflare analytics and logs
+
+**It should work!** 🚀 (And now we have a plan if it doesn't!)
+
+## Advanced: Securing Administrative Access
+
+### Securing Kubernetes & Talos APIs
+
+Once application migration is complete, you can secure administrative access:
+
+#### Option 1: TCP Proxy (Simpler)
+```yaml
+# Cloudflare Zero Trust → Tunnels → Configure
+Public Hostname:
+  Subdomain: api
+  Domain: keyboardvagabond.com
+  Service: tcp://localhost:6443  # Kubernetes API
+  
+Public Hostname:
+  Subdomain: talos  
+  Domain: keyboardvagabond.com
+  Service: tcp://<NODE_1_IP>:50000  # Talos API
+```
+
+**Client configuration:**
+```bash
+# Update kubectl config
+kubectl config set-cluster keyboardvagabond \
+  --server=https://api.keyboardvagabond.com:443  # Note: 443, not 6443
+
+# Update talosctl config  
+talosctl config endpoint talos.keyboardvagabond.com:443
+```
+
+#### Option 2: Private Network via WARP (Most Secure)
+
+**Step 1: Configure Private Network**
+```yaml
+# Cloudflare Zero Trust → Tunnels → Configure → Private Networks
+Private Network:
+  CIDR: 10.132.0.0/24  # Your NetCup vLAN network
+  Description: "Keyboard Vagabond Cluster Internal Network"
+```
+
+**Step 2: Configure Split Tunnels**  
+```yaml
+# Zero Trust → Settings → WARP Client → Device settings → Split Tunnels
+Mode: Exclude (recommended)
+Remove: 10.0.0.0/8  # Remove broad private range
+Add back:
+  - 10.0.0.0/9      # 10.0.0.0 - 10.127.255.255  
+  - 10.133.0.0/16   # 10.133.0.0 - 10.133.255.255
+  - 10.134.0.0/15   # 10.134.0.0 - 10.135.255.255
+  # This ensures only 10.132.0.0/24 routes through WARP
+```
+
+**Step 3: Client Configuration**
+```bash
+# Install WARP client on admin machines
+# macOS: brew install --cask cloudflare-warp
+# Connect to Zero Trust organization
+warp-cli registration new
+
+# Configure kubectl to use internal IPs
+kubectl config set-cluster keyboardvagabond \
+  --server=https://<NODE_1_IP>:6443  # Direct to internal node IP
+
+# Configure talosctl to use internal IPs  
+talosctl config endpoint <NODE_1_IP>:50000,<NODE_2_IP>:50000
+```
+
+**Step 4: Access Policies (Recommended)**
+```yaml
+# Zero Trust → Access → Applications → Add application
+Application Type: Private Network
+Name: "Kubernetes Cluster Admin Access"
+Application Domain: 10.132.0.0/24
+
+Policies:
+  - Name: "Admin Team Only"
+    Action: Allow
+    Rules: 
+      - Email domain: @yourdomain.com
+      - Device Posture: Managed device required
+```
+
+**Step 5: Device Enrollment**
+```bash
+# On admin device
+# 1. Install WARP: https://1.1.1.1/
+# 2. Login with Zero Trust organization  
+# 3. Verify private network access:
+ping <NODE_1_IP>  # Should work through WARP
+
+# 4. Test API access
+kubectl get nodes  # Should connect to internal cluster
+talosctl version   # Should connect to internal Talos API
+```
+
+**Step 6: Lock Down External Access**
+Once WARP is working, update Talos machine configs to block external access:
+```yaml
+# In machineconfigs/n1.yaml and n2.yaml
+machine:
+  network:
+    extraHostEntries:
+      # Firewall rules via Talos
+      - ip: 127.0.0.1  # Placeholder - actual firewall config needed
+```
+
+#### WARP Benefits:
+- ✅ **No public DNS entries** - Admin endpoints not discoverable  
+- ✅ **Device control** - Only managed devices can access cluster
+- ✅ **Zero-trust policies** - Granular access control per user/device
+- ✅ **Audit logs** - Full visibility into who accessed what when
+- ✅ **Device posture** - Require encryption, OS updates, etc.
+- ✅ **Split tunneling** - Only cluster traffic goes through tunnel
+- ✅ **Automatic failover** - Multiple WARP data centers
+
+## Testing WARP Implementation
+
+### Before WARP (Current State)
+```bash
+# Current kubectl configuration
+kubectl config view --minify -o jsonpath='{.clusters[0].cluster.server}'
+# Output: https://api.keyboardvagabond.com:6443
+
+# This goes through internet → external IPs
+kubectl get nodes
+```
+
+### After WARP Setup
+```bash
+# 1. Test private network connectivity first
+ping <NODE_1_IP>  # Should work once WARP is connected
+
+# 2. Create backup kubectl context  
+kubectl config set-context keyboardvagabond-external \
+  --cluster=keyboardvagabond.com \
+  --user=admin@keyboardvagabond.com
+
+# 3. Update main context to use internal IP
+kubectl config set-cluster keyboardvagabond.com \
+  --server=https://<NODE_1_IP>:6443
+
+# 4. Test internal access
+kubectl get nodes  # Should work through WARP → private network
+
+# 5. Verify traffic path
+# WARP status should show "Connected" in system tray
+warp-cli status  # Should show connected to your Zero Trust org
+```
+
+### Rollback Plan
+```bash
+# If WARP doesn't work, quickly restore external access:
+kubectl config set-cluster keyboardvagabond.com \
+  --server=https://api.keyboardvagabond.com:6443
+
+# Test external access still works
+kubectl get nodes
+```
+
+## Next Steps After WARP
+
+Once WARP is proven working:
+1. **Configure Talos firewall** to block external access to ports 6443 and 50000
+2. **Remove public API DNS entry** (api.keyboardvagabond.com)  
+3. **Document emergency access procedure** (temporary firewall rule + external DNS)
+4. **Set up additional WARP devices** for other administrators
+
+This gives you a **zero-trust administrative access model** where cluster APIs are completely invisible from the internet! 🔒