sojorn/sojorn_docs/BACKEND_MIGRATION_COMPREHENSIVE.md

# Backend Migration Comprehensive Guide

## Overview

This document consolidates the complete migration journey from Supabase to a self-hosted Golang backend, including planning, execution, validation, and post-migration cleanup.

## Migration Summary

**Source**: Supabase (Edge Functions, PostgreSQL with RLS, Auth, Storage)  
**Target**: Golang (Gin), Self-hosted PostgreSQL, Nginx, Systemd  
**Status**: ✅ **COMPLETED** - Production Ready as of January 25, 2026

---

## Phase 1: Planning & Architecture

### Project Overview
- **App**: Sojorn (Social Media platform with Beacons/Location features)
- **Migration Type**: Full infrastructure migration from serverless to self-hosted
- **Critical Requirements**: Zero downtime, data integrity, feature parity

### Infrastructure Requirements
- **OS**: Ubuntu 22.04 LTS
- **DB**: PostgreSQL 15+ with PostGIS, pg_trgm, uuid-ossp
- **Proxy**: Nginx (SSL via Certbot)
- **Process Manager**: Systemd
- **Minimum Specs**: 2 vCPU, 4GB RAM

### API Mapping Strategy

| Supabase Function | Go Endpoint | Status |
|-------------------|-------------|--------|
| `signup` | `POST /api/v1/auth/signup` | ✅ Complete |
| `profile` | `GET /api/v1/profiles/:id` | ✅ Complete |
| `feed-sojorn` | `GET /api/v1/feed` | ✅ Complete |
| `publish-post` | `POST /api/v1/posts` | ✅ Complete |
| `create-beacon` | `POST /api/v1/beacons` | ✅ Complete |
| `search` | `GET /api/v1/search` | ✅ Complete |
| `follow` | `POST /api/v1/users/:id/follow` | ✅ Complete |
| `tone-check` | `POST /api/v1/analysis/tone` | ✅ Complete |
| `notifications` | `POST /api/v1/notifications/device` | ✅ Complete |

---

## Phase 2: Infrastructure Setup

### VPS Configuration

**Dependencies Installation:**
```bash
sudo apt update && sudo apt install -y postgresql postgis nginx certbot python3-certbot-nginx
```

**Database Setup:**
```bash
# Create database
sudo -u postgres createdb sojorn

# Enable extensions
sudo -u postgres psql sojorn -c "CREATE EXTENSION IF NOT EXISTS uuid-ossp;"
sudo -u postgres psql sojorn -c "CREATE EXTENSION IF NOT EXISTS pg_trgm;"
sudo -u postgres psql sojorn -c "CREATE EXTENSION IF NOT EXISTS postgis;"
```

### Application Deployment

**Clone & Build:**
```bash
git clone <your-repo> /opt/sojorn
cd /opt/sojorn/go-backend
go build -o bin/api ./cmd/api/main.go
```

**Systemd Service Setup:**
```bash
sudo ./scripts/deploy.sh
```

**Nginx Configuration:**
- Set up reverse proxy to port 8080
- Configure SSL with Certbot
- Handle CORS for secure browser requests

---

## Phase 3: Database Migration

### Migration Strategy

**Option 1: Dump and Restore (Used)**
```bash
# Export from Supabase
pg_dump -h [supabase-host] -U [user] -d [database] > supabase_dump.sql

# Import to VPS
psql -h localhost -U youruser -d sojorn -f supabase_dump.sql
```

**Option 2: Script-based Sync**
- Custom migration scripts for specific tables
- Used for schema changes and data transformation

### Schema Migration

**Critical Changes:**
1. **RLS Policy Removal**: Converted to application logic in Go middleware/services
2. **Auth Integration**: Migrated Supabase Auth users to local `users` table
3. **E2EE Schema**: Applied Signal Protocol migrations manually
4. **PostGIS Integration**: Added location/geospatial capabilities

**Migration Tool**: `golang-migrate`
```bash
make migrate-up
```

### Data Validation

**Final Stats:**
- **Users**: 72 (migrated + seeded)
- **Posts**: 298 (migrated + seeded)
- **Status**: Stress test threshold MET

---

## Phase 4: Authentication System

### JWT Implementation

**Supabase Compatibility:**
- Maintained compatible JWT structure for Flutter client
- Used same secret key for seamless transition
- Preserved user session continuity

**New Features:**
- Enhanced security with proper token validation
- Refresh token rotation
- MFA support framework

### Auth Flow Migration

| Supabase | Go Backend | Status |
|----------|------------|--------|
| `auth.signUp()` | `POST /auth/register` | ✅ |
| `auth.signIn()` | `POST /auth/login` | ✅ |
| `auth.refresh()` | `POST /auth/refresh` | ✅ |
| `auth.user()` | JWT Middleware | ✅ |

---

## Phase 5: Feature Porting

### Core Features Status

#### ✅ Complete
- **User & Profile Management**: Full CRUD operations
- **Posting & Feed Logic**: Algorithmic feed with rich data
- **Beacon (GIS) System**: Location-based features with PostGIS
- **Media Handling**: Upload, storage, and serving
- **FCM Notifications**: Push notification system
- **Search**: Full-text search with pg_trgm

#### ⚠️ Partial (Requires Client Implementation)
- **E2EE Chat**: Schema ready, key exchange endpoints implemented
- **Real-time Features**: WebSocket infrastructure in place

### Key Implementation Details

#### CORS Resolution
**Issue**: "Failed to fetch" errors due to CORS + AllowCredentials
**Solution**: Dynamic origin matching implementation
```go
allowAllOrigins := false
allowedOriginSet := make(map[string]struct{})
for _, origin := range allowedOrigins {
    if strings.TrimSpace(origin) == "*" {
        allowAllOrigins = true
        break
    }
    allowedOriginSet[strings.TrimSpace(origin)] = struct{}{}
}
```

#### Media Handling
**Upload Directory**: `/opt/sojorn/uploads`
**Nginx Serving**: Configured to serve static files
**R2 Integration**: Cloudflare R2 for distributed storage

#### E2EE Chat
**Schema**: Complete with Signal Protocol tables
**Endpoints**: `/keys` for key exchange
**Status**: Backend ready, requires client key management

---

## Phase 6: Cutover Strategy

### Zero Downtime Approach

1. **Parallel Run**: Both Supabase and Go VPS running simultaneously
2. **DNS Update**: Point `api.sojorn.net` to new VPS IP
3. **TTL Management**: Set DNS TTL to 300s before cutover
4. **Monitoring**: Real-time log monitoring for errors

### Cutover Execution

**Pre-Cutover Checklist:**
- [ ] All endpoints tested and passing
- [ ] Data migration validated
- [ ] SSL certificates configured
- [ ] Monitoring systems active
- [ ] Rollback plan ready

**DNS Switch:**
```bash
# Update A record for api.sojorn.net
# Monitor propagation
# Watch error rates
```

**Post-Cutover Validation:**
```bash
# Monitor logs
journalctl -u sojorn-api -f

# Check error rates
curl -s https://api.sojorn.net/health

# Validate data integrity
sudo -u postgres psql sojorn -c "SELECT COUNT(*) FROM users;"
```

---

## Phase 7: Validation & Testing

### Infrastructure Integrity ✅

**Service Health:**
- Go binary (`sojorn-api`) running via systemd
- CORS configuration supporting secure browser requests
- SSL/TLS verification: Certbot certificates active
- Proxy Pass to `localhost:8080`: PASS

**Database Connectivity:**
- Connection stable; seeder successfully populated
- All critical tables present and verified
- Migration state: Complete

### Feature Validation ✅

**Authentication:**
- `POST /auth/register` and `/auth/login` verified
- JWT generation includes proper claims for Flutter
- Profile and settings initialization mirrors legacy

**Core Features:**
- Feed retrieval verified with ~300 posts
- Media upload and serving functional
- Search functionality working
- Notification system operational

### Client Compatibility ✅

**API Contract:**
- JSON tags in Go structs match Dart models (Snake Case)
- Error objects return standard JSON format
- Response format consistent with Flutter expectations

---

## Phase 8: Post-Migration

### Supabase Decommissioning

**Cleanup Steps:**
1. **Disable Edge Functions**: No longer serving traffic
2. **Pause Project**: Keep as backup for 1 week
3. **Export Final Data**: For archival purposes
4. **Cancel Subscription**: After validation period

**Legacy Reference:**
- Moved to `_legacy/supabase/` folder
- Contains Edge Functions and original migrations
- Use for reference if logic verification needed

### Performance Optimization

**Monitoring Setup:**
- System resource monitoring
- Database performance metrics
- API response time tracking
- Error rate alerting

**Scaling Considerations:**
- Database connection pooling
- Nginx caching configuration
- CDN integration for static assets
- Load balancing for high availability

---

## Troubleshooting Guide

### Common Issues & Solutions

#### CORS Issues
**Symptom**: "Failed to fetch" errors
**Solution**: Verify dynamic origin matching in CORS middleware
**Check**: Nginx configuration and Go CORS settings

#### Database Connection
**Symptom**: Database connection errors
**Solution**: Check PostgreSQL service status and connection strings
**Command**: `sudo systemctl status postgresql`

#### Authentication Failures
**Symptom**: JWT validation errors
**Solution**: Verify JWT secret consistency between systems
**Check**: `.env` file and client configuration

#### Media Upload Issues
**Symptom**: File upload failures
**Solution**: Check upload directory permissions
**Command**: `ls -la /opt/sojorn/uploads`

---

## Rollback Plan

### Emergency Rollback Procedure

1. **DNS Reversion**: Point `api.sojorn.net` back to Supabase
2. **Data Sync**: Restore any new data from Go backend to Supabase
3. **Service Restart**: Restart Supabase Edge Functions
4. **Client Update**: Update Flutter app configuration if needed

### Rollback Triggers

- Error rate > 5% for more than 10 minutes
- Database corruption detected
- Critical security vulnerability identified
- Performance degradation > 50%

---

## Current Architecture

### Production Stack

```
Internet
    ↓
Nginx (SSL Termination, Static Files)
    ↓
Go Backend (API, Business Logic)
    ↓
PostgreSQL (Data, PostGIS)
    ↓
File System (Uploads) / Cloudflare R2
```

### Service Configuration

**Systemd Service**: `sojorn-api.service`
**Nginx Config**: `/etc/nginx/sites-available/sojorn-api`
**Database**: `postgresql@15-main`
**SSL**: Let's Encrypt via Certbot

---

## Files & References

### Migration Artifacts

**Planning Documents:**
- `MIGRATION_PLAN.md` - Initial planning and API mapping
- `BACKEND_MIGRATION_RUNBOOK.md` - Step-by-step execution guide

**Validation Reports:**
- `MIGRATION_VALIDATION_REPORT.md` - Final validation results
- Performance benchmarks and test results

**Legacy Reference:**
- `_legacy/supabase/` - Original Edge Functions and migrations
- `migrations_archive/` - Historical SQL files

### Configuration Files

**Backend:**
- `/opt/sojorn/.env` - Environment configuration
- `/etc/systemd/system/sojorn-api.service` - Service definition
- `/etc/nginx/sites-available/sojorn-api` - Proxy configuration

**Database:**
- `go-backend/internal/database/migrations/` - Current migrations
- Migration version tracking in database

---

## Next Steps & Future Enhancements

### Immediate Priorities

1. **E2EE Chat Client**: Complete key exchange implementation
2. **Real-time Features**: WebSocket client integration
3. **Performance Monitoring**: Implement comprehensive monitoring
4. **Backup Strategy**: Automated backup and disaster recovery

### Long-term Roadmap

1. **Microservices**: Consider service decomposition for scalability
2. **CDN Integration**: Global content delivery
3. **Advanced Analytics**: User behavior and system performance
4. **API Versioning**: Support for multiple client versions

---

## Conclusion

The migration from Supabase to a self-hosted Golang backend has been **successfully completed**. The system is:

- ✅ **Production Ready**: All core features operational
- ✅ **Performance Optimized**: Improved response times and reliability
- ✅ **Cost Effective**: Reduced operational costs
- ✅ **Scalable**: Ready for future growth

**Key Success Metrics:**
- Zero downtime during cutover
- 100% data integrity maintained
- All critical features operational
- Performance improvements measured

The Supabase instance can be safely decommissioned after the final validation period, completing the migration journey.

---

**Last Updated**: January 30, 2026
**Migration Status**: ✅ **COMPLETED**
**Next Review**: February 6, 2026