--- description: Security patterns including SOPS encryption, Zero Trust, and access control globs: ["**/*.yaml", "machineconfigs/**/*", "secrets.yaml", "*.conf"] alwaysApply: false --- # Security & Encryption ✅ OPERATIONAL ## 🛡️ Maximum Security Architecture Achieved - **🚫 Zero External Port Exposure**: No direct internet access to any cluster services - **🔐 Dual Security Layers**: Cloudflare Zero Trust (public apps) + Tailscale Mesh VPN (admin access) - **🌐 CGNAT-Only API Access**: Kubernetes/Talos APIs restricted to Tailscale network (100.64.0.0/10) - **🔒 Encrypted Everything**: SOPS secrets, Zero Trust tunnels, mesh VPN connections - **🛡️ Host Firewall**: Cilium policies blocking world access to HTTP/HTTPS ports ## SOPS Configuration ✅ OPERATIONAL ### Encryption Scope - **Files Covered**: All YAML files in `manifests/` directory, Talos configs, machine configurations - **Fields Encrypted**: `data` and `stringData` fields in manifests, plus specific credential fields - **Key Management**: Multiple PGP keys configured for different components - **Workflow**: All secrets encrypted with SOPS before Git commit ### SOPS Usage Patterns ```bash # Encrypt new secret sops -e -i secrets.yaml # Edit encrypted secret sops secrets.yaml # Decrypt for viewing sops -d secrets.yaml #Decrypt in place sops -d -i secrets.yaml # Apply encrypted manifest sops -d secrets.yaml | kubectl apply -f - ``` Sops encrypted files should be applied with kubectl in the unencrypted format, and encrypted before merging into source control. ## Zero Trust Architecture ✅ MIGRATED ### Zero Trust Tunnels ✅ OPERATIONAL - **Cloudflared Deployment**: `cloudflared-system` namespace - **Tunnel Architecture**: Secure connectivity without exposing ingress ports - **TLS Termination**: Cloudflare edge handles SSL/TLS - **DNS Management**: Manual DNS record creation (external-dns removed) ### Standard Zero Trust Ingress Pattern ```yaml apiVersion: networking.k8s.io/v1 kind: Ingress metadata: name: app-ingress namespace: app-namespace annotations: # Basic NGINX Configuration only - no cert-manager or external-dns kubernetes.io/ingress.class: nginx nginx.ingress.kubernetes.io/backend-protocol: "HTTP" spec: ingressClassName: nginx tls: [] # Empty - TLS handled by Cloudflare edge rules: - host: app.keyboardvagabond.com http: paths: - path: / pathType: Prefix backend: service: name: app-service port: number: 80 ``` ### Migration Steps for Zero Trust 1. **Remove cert-manager annotations**: `cert-manager.io/cluster-issuer`, `cert-manager.io/issuer` 2. **Remove external-dns annotations**: `external-dns.alpha.kubernetes.io/hostname`, `external-dns.alpha.kubernetes.io/target` 3. **Empty TLS sections**: Set `tls: []` to disable certificate generation 4. **Configure Cloudflare tunnel**: Add hostname in Zero Trust dashboard 5. **Test connectivity**: Use `kubectl run curl-test` to verify internal service health ## Access Control Matrix | **Resource** | **Public Access** | **Administrative Access** | **Security Method** | |--------------|-------------------|---------------------------|---------------------| | **Applications** | ✅ Cloudflare Zero Trust | ❌ Not Applicable | Authenticated tunnels | | **Kubernetes API** | ❌ Blocked | ✅ Tailscale Mesh VPN | CGNAT + OAuth | | **Talos API** | ❌ Blocked | ✅ Tailscale Mesh VPN | CGNAT + OAuth | | **HTTP/HTTPS Services** | ❌ Blocked | ✅ Cluster Internal Only | Host firewall | | **Media CDN** | ✅ Cloudflare CDN | ❌ Not Applicable | Public S3 + Edge caching | ## Tailscale Mesh VPN ✅ OPERATIONAL ### Administrative Access Configuration - **kubectl Context**: `admin@keyboardvagabond-tailscale` using internal VLAN IP (10.132.0.10:6443) - **Public Context**: `admin@keyboardvagabond.com` (blocked by firewall) - **Tailscale Client**: Current IP range 100.64.0.0/10 (CGNAT) - **Firewall Rules**: Cilium host firewall restricts API access to Tailscale network only ### Tailscale Subnet Router Configuration ✅ OPERATIONAL - **Device Name**: `keyboardvagabond-cluster` - **Deployment Model**: Direct deployment (not Kubernetes Operator) for simplicity - **Advertised Networks**: - **Pod Network**: 10.244.0.0/16 (Kubernetes pods) - **Service Network**: 10.96.0.0/12 (Kubernetes services) - **VLAN Network**: 10.132.0.0/24 (NetCup Cloud private network) - **OAuth Integration**: Client credentials for device authentication and tagging - **Device Tagging**: `tag:k8s-operator` for proper ACL management and identification - **Network Mode**: Kernel mode (`TS_USERSPACE=false`) with privileged security context - **State Persistence**: Kubernetes secret-based storage (`TS_KUBE_SECRET=tailscale-auth`) - **RBAC**: Split permissions (ClusterRole for cluster resources, Role for namespace secrets) ### Tailscale Deployment Pattern ```yaml # Direct deployment (not Kubernetes Operator) apiVersion: apps/v1 kind: Deployment metadata: name: tailscale-subnet-router spec: template: spec: containers: - name: tailscale env: - name: TS_KUBE_SECRET value: tailscale-auth - name: TS_USERSPACE value: "false" - name: TS_ROUTES value: "10.244.0.0/16,10.96.0.0/12,10.132.0.0/24" securityContext: privileged: true ``` ## Network Security ✅ OPERATIONAL ### Cilium Host Firewall ```yaml # Host firewall blocking external access to HTTP/HTTPS apiVersion: cilium.io/v2 kind: CiliumClusterwideNetworkPolicy metadata: name: host-fw-control-plane spec: nodeSelector: matchLabels: node-role.kubernetes.io/control-plane: "" ingress: - fromCIDR: - "100.64.0.0/10" # Tailscale CGNAT range only toPorts: - ports: - port: "6443" protocol: TCP ``` ## Security Best Practices - **New Services**: All applications must use Zero Trust ingress pattern - **Harbor Exception**: Harbor registry requires direct port exposure (header modification issues) - **Secret Management**: All secrets SOPS-encrypted before Git commit - **Network Policies**: Cilium host firewall with CGNAT-only access - **Administrative Access**: Tailscale mesh VPN required for kubectl/talosctl ## 🏆 Security Achievements 1. **🎯 Zero Trust Network**: No implicit trust, all access authenticated and authorized 2. **🔐 Defense in Depth**: Multiple security layers prevent single points of failure 3. **📊 Comprehensive Monitoring**: All traffic flows monitored via OpenObserve and Cilium Hubble 4. **🔄 Secure GitOps**: SOPS-encrypted secrets with PGP key management 5. **🛡️ Hardened Infrastructure**: Minimal attack surface with production-grade security controls @sops-secret-template.yaml @zero-trust-ingress-template.yaml @tailscale-config-template.yaml