Keybard-Vagabond-Demo/build/postgresql-postgis/Migration_Plan.md

# PostgreSQL 18 + PostGIS 3.6 for CloudNativePG (ARM64 Source Build)

## Overview

Upgrade from PostgreSQL 16 to PostgreSQL 18 with PostGIS 3.6 for ARM64 CloudNativePG deployment.

**Why build from source?**
- The official `postgis/postgis:18-3.6` image only supports AMD64, not ARM64
- `imresamu/postgis` hasn't released PG18 ARM64 images yet
- Building from source ensures ARM64 compatibility for your cluster

**Current Setup:**
- Image: `registry.keyboardvagabond.com/library/cnpg-postgis:16.6-3.4-v2`
- Base: `imresamu/postgis:16-3.4`
- PostgreSQL: 16.6
- PostGIS: 3.4

**Target Setup:**
- Image: `registry.keyboardvagabond.com/library/cnpg-postgis:18-3.6`
- Base: `postgres:18-bookworm` (official, ARM64 supported)
- PostgreSQL: 18.1
- PostGIS: 3.6.1 (compiled from source)
- GEOS: 3.13.0
- PROJ: 9.4.1
- GDAL: 3.10.1

## Extensions Included

| Extension | Description | Status |
|-----------|-------------|--------|
| postgis | Core GIS functionality | ✓ Compiled |
| postgis_topology | Topology support | ✓ Compiled |
| postgis_raster | Raster support | ✓ Compiled |
| fuzzystrmatch | Fuzzy string matching | ✓ Compiled |
| postgis_tiger_geocoder | US Census TIGER geocoder | ✓ Compiled |

## Build Architecture

```
┌─────────────────────────────────────────────────────────────────┐
│ Stage 1: Builder                                                │
│ - Base: postgres:18-bookworm (ARM64)                           │
│ - Compile GEOS 3.13.0                                          │
│ - Compile PROJ 9.5.1                                           │
│ - Compile GDAL 3.10.1                                          │
│ - Compile PostGIS 3.6.1                                        │
├─────────────────────────────────────────────────────────────────┤
│ Stage 2: CNPG Tools                                             │
│ - ghcr.io/cloudnative-pg/postgresql:18                         │
│ - Extract barman-cloud tools for backup/restore                │
├─────────────────────────────────────────────────────────────────┤
│ Stage 3: Final Image (minimal)                                  │
│ - Base: postgres:18-bookworm (ARM64)                           │
│ - Copy compiled libs from Stage 1                              │
│ - Copy barman tools from Stage 2                               │
│ - Fix postgres UID to 26 (CNPG requirement)                    │
│ - Runtime dependencies only                                     │
└─────────────────────────────────────────────────────────────────┘
```

## ⚠️ Important: PG18 Data Directory Change

PostgreSQL 18 changed the default data directory path:

| Version | Data Directory |
|---------|----------------|
| PG 13-17 | `/var/lib/postgresql/data` |
| PG 18+ | `/var/lib/postgresql` |

This affects volume mounts in docker-compose and may require CNPG configuration changes.

## Migration Steps

### Phase 1: Build and Test Locally

1. **Build the image (takes 15-30 minutes):**
   ```bash
   cd build/postgresql-postgis
   chmod +x build.sh initdb-postgis.sh update-postgis.sh
   ./build.sh
   ```

2. **Test with docker-compose:**
   ```bash
   docker-compose -f docker-compose.test.yaml up -d
   docker-compose -f docker-compose.test.yaml exec postgres psql -U postgres
   
   # In psql, verify:
   SELECT postgis_full_version();
   SELECT ST_AsText(ST_Point(0, 0));
   \dx  -- list extensions
   
   # Cleanup
   docker-compose -f docker-compose.test.yaml down -v
   ```

3. **Interactive testing:**
   ```bash
   docker run -it --rm -e POSTGRES_PASSWORD=test cnpg-postgis:18-3.6 bash
   ```

### Phase 2: Push to Registry

```bash
docker push registry.keyboardvagabond.com/library/cnpg-postgis:18-3.6
```

### Phase 3: CNPG Upgrade

**Option A: In-place upgrade (for testing)**

1. Update `manifests/infrastructure/postgresql/cluster-shared.yaml`:
   ```yaml
   spec:
     imageName: registry.keyboardvagabond.com/library/cnpg-postgis:18-3.6
   ```

2. CNPG will handle the rolling upgrade automatically.

**Option B: Create new cluster and migrate (safer for production)**

1. Create a new cluster with PG18 image
2. Use pg_dump/pg_restore or CNPG backup/restore
3. Switch applications to new cluster
4. Decommission old cluster

## CNPG Operator Compatibility

- Current operator: `>=0.20.0` (Helm chart)
- PostgreSQL 18 support: Requires CNPG operator 1.24+
- Check current version:
  ```bash
  kubectl get deployment -n postgresql-system -l app.kubernetes.io/name=cloudnative-pg \
    -o jsonpath='{.items[0].spec.template.spec.containers[0].image}'
  ```

If upgrade needed, update `manifests/infrastructure/postgresql/operator.yaml`:
```yaml
spec:
  chart:
    spec:
      version: ">=0.23.0"  # or specific version with PG18 support
```

## Rollback Plan

If issues occur:
1. Change imageName back to `registry.keyboardvagabond.com/library/cnpg-postgis:16.6-3.4-v2`
2. CNPG will roll back to previous version
3. Restore from backup if data issues

## Verification Checklist

- [ ] Image builds successfully on M1 Mac (~15-30 min)
- [ ] postgres user has UID 26
- [ ] GEOS, PROJ, GDAL compiled correctly
- [ ] PostGIS extensions install correctly
- [ ] barman-cloud tools are present
- [ ] Local docker-compose test passes
- [ ] Spatial queries work (`ST_Point`, `ST_AsText`, etc.)
- [ ] Image pushed to Harbor registry
- [ ] CNPG operator compatible with PG18
- [ ] Test cluster upgrade in staging (if available)
- [ ] Production cluster upgrade successful
- [ ] All fediverse apps functioning correctly

## Build Dependencies (compiled from source)

| Library | Version | Purpose |
|---------|---------|---------|
| GEOS | 3.13.0 | Geometry operations |
| PROJ | 9.4.1 | Coordinate transformations |
| GDAL | 3.10.1 | Raster/vector data access |
| PostGIS | 3.6.1 | PostgreSQL spatial extension |

## References

- [PostgreSQL 18 Release Notes](https://www.postgresql.org/docs/18/release-18.html)
- [PostGIS 3.6 Release Notes](https://postgis.net/documentation/getting_started/)
- [docker-postgis GitHub](https://github.com/postgis/docker-postgis)
- [CloudNativePG Documentation](https://cloudnative-pg.io/documentation/)
- [GEOS Downloads](https://download.osgeo.org/geos/)
- [PROJ Downloads](https://download.osgeo.org/proj/)
- [GDAL Downloads](https://github.com/OSGeo/gdal/releases)