# End-to-End Encryption (E2EE) Comprehensive Guide

## Overview

This document consolidates all E2EE implementation knowledge for Sojorn, covering the complete evolution from simple stateless encryption to the current X3DH-based production system.

## Current Architecture (Production System)

### Cryptographic Foundation
- **Flutter Client**: Uses X25519 for key exchange, Ed25519 for signatures, AES-GCM for encryption
- **Go Backend**: Stores key bundles in PostgreSQL, serves encryption keys
- **Protocol**: X3DH (Extended Triple Diffie-Hellman) for key agreement

### Key Components

#### 1. Key Storage
- **FlutterSecureStorage**: Local key persistence with `e2ee_keys_v3` key
- **PostgreSQL Tables**: `profiles`, `signed_prekeys`, `one_time_prekeys`
- **Key Format**: Identity keys stored as `Ed25519:X25519` (base64 concatenated with colon)

#### 2. Key Generation Flow
1. Generate Ed25519 signing key pair (for signatures)
2. Generate X25519 identity key pair (for DH)
3. Generate X25519 signed prekey with Ed25519 signature
4. Generate 20 X25519 one-time prekeys (OTKs)
5. Upload key bundle to backend

#### 3. Message Encryption Flow
1. Fetch recipient's key bundle from backend
2. Verify signed prekey signature with Ed25519
3. Perform X3DH key agreement
4. Derive shared secret using KDF (SHA-256)
5. Encrypt message with AES-GCM
6. Delete used OTK from server

## Historical Evolution

### Phase 1: Simple Stateless E2EE (Legacy)

**Description**: Basic stateless system using X25519 + AES-GCM with single static identity keys.

**Architecture**:
- Each user had a single static identity key pair
- Each message used a fresh ephemeral key pair
- Shared secret derived via X25519 ECDH
- Sender could not decrypt their own message history

**Data Model**:
```
profiles.identity_key (base64 X25519 public key)
encrypted_conversations (conversation metadata)
encrypted_messages (ciphertext + header + metadata)
```

**Message Header Format**:
```json
{
  "epk": "<base64 sender ephemeral public key>",
  "n":   "<base64 nonce>",
  "m":   "<base64 MAC>",
  "v":   1
}
```

**Limitations**:
- No forward secrecy beyond individual messages
- No multi-device support
- Senders couldn't decrypt their own message history
- No key recovery mechanism

### Phase 2: X3DH Implementation (Current)

**Description**: Full X3DH implementation with signed prekeys, one-time prekeys, and proper key management.

**Improvements**:
- ✅ Perfect Forward Secrecy via OTKs
- ✅ Post-Compromise Security via key rotation
- ✅ Authentication via Ed25519 signatures
- ✅ Confidentiality via AES-GCM
- ✅ Cross-platform compatibility (Android↔Web)
- ✅ Automatic key management

## Issues Encountered & Resolutions

### Issue #1: 208-bit Key Bug ❌→✅
**Problem**: Keys were 26 characters (208 bits) instead of 32 bytes (256 bits)
**Root Cause**: Using string-based KDF instead of proper byte-based KDF
**Fix**: Updated `_kdf` method to use SHA-256 on byte arrays
**Files Modified**: `simple_e2ee_service.dart`

### Issue #2: Database Constraint Error ❌→✅
**Problem**: `SQLSTATE 42P10` - ON CONFLICT constraint mismatch
**Root Cause**: Go code used `ON CONFLICT (user_id)` but DB had `PRIMARY KEY (user_id, key_id)`
**Fix**: Updated Go code to use correct constraint `ON CONFLICT (user_id, key_id)`
**Files Modified**: `user_repository.go`

### Issue #3: Fake Zero Signatures ❌→✅
**Problem**: SPK signatures were all zeros (`AAAAAAAA...`)
**Root Cause**: Manual upload used fake signature for testing
**Fix**: Updated manual upload to generate real Ed25519 signatures
**Files Modified**: `simple_e2ee_service.dart`

### Issue #4: Asymmetric Security ❌→✅
**Problem**: One user skipped signature verification (legacy), other enforced it
**Root Cause**: Legacy user detection created security asymmetry
**Fix**: Removed legacy logic, enforced signature verification for all users
**Files Modified**: `simple_e2ee_service.dart`

### Issue #5: Key Upload Not Automatic ❌→✅
**Problem**: Keys loaded locally but never uploaded to backend
**Root Cause**: `_doInitialize` returned early after loading keys
**Fix**: Added backend existence check and automatic upload
**Files Modified**: `simple_e2ee_service.dart`

### Issue #6: NULL Database Values ❌→✅
**Problem**: `registration_id` was NULL causing scan errors
**Root Cause**: Database column allowed NULL values
**Fix**: Updated Go code to handle `sql.NullInt64` with default values
**Files Modified**: `user_repository.go`

### Issue #7: Noisy WebSocket Logs ❌→✅
**Problem**: Ping/pong messages cluttered console
**Root Cause**: WebSocket heartbeat logging
**Fix**: Filtered out ping/pong messages completely
**Files Modified**: `secure_chat_service.dart`

### Issue #8: Modal Header Override ❌→✅
**Problem**: AppBar changes in chat screen were hidden by modal wrapper
**Root Cause**: `SecureChatModal` had custom header overriding `SecureChatScreen` AppBar
**Fix**: Added upload button to modal header instead
**Files Modified**: `secure_chat_modal_sheet.dart`

## Current Status ✅

### Working Components
- ✅ 32-byte key generation
- ✅ Valid Ed25519 signatures
- ✅ Signature verification
- ✅ Key bundle upload/download
- ✅ X3DH key agreement
- ✅ AES-GCM encryption/decryption
- ✅ OTK management (generation, usage, deletion)
- ✅ Backend key storage/retrieval
- ✅ Cross-platform encryption (Android↔Web)
- ✅ **Full Backup & Recovery** (Keys + Messages)

### Key Files Modified
```
Flutter:
- lib/services/simple_e2ee_service.dart (core E2EE logic)
- lib/services/secure_chat_service.dart (WebSocket + key management)
- lib/screens/secure_chat/secure_chat_modal_sheet.dart (UI upload button)

Go Backend:
- internal/handlers/key_handler.go (API endpoints + validation)
- internal/repository/user_repository.go (database operations)
```

### Database Schema
```sql
-- Key storage tables
profiles (identity_key, registration_id)
signed_prekeys (user_id, key_id, public_key, signature)
one_time_prekeys (user_id, key_id, public_key)
```

## Testing Checklist

### Before Testing
1. Ensure both users have valid keys (check `[E2EE] Keys exist on backend - ready`)
2. Verify signatures are non-zero (check backend logs)
3. Confirm OTKs are available (should have 20 OTKs each)

### Test Flow
1. **Key Upload**: Tap "🔑" button → should see `[E2EE] Key bundle uploaded successfully`
2. **Message Send**: Type message → should see `[E2EE] SPK signature verified successfully`
3. **Message Receive**: Should see `[DECRYPT] SUCCESS: Decrypted message: "..."`
4. **OTK Deletion**: Should see `[E2EE] Deleted used OTK #[id] from server`

### Expected Logs
```
Sender:
[ENCRYPT] Fetching key bundle for recipient: [...]
[E2EE] SPK signature verified successfully.
[E2EE] Deleted used OTK #[id] from server

Receiver:
[DECRYPT] Used OTK with key_id: [id]
[DECRYPT] SUCCESS: Decrypted message: "[message_text]"
```

## Backup & Recovery System ✅

### Overview
A robust local backup and recovery system has been implemented to address the risk of data loss on device changes or app uninstalls. This system allows users to export their cryptographic identity and message history into a secure, portable file.

### Architecture

#### 1. Security Model
- **Encryption**: AES-256-GCM
- **Key Derivation**: Argon2id (from user password)
- **Storage**: Local file system (portable JSON file)
- **Trust**: Zero-knowledge (server never sees the backup file or password)

#### 2. Backup Content
The encrypted backup file contains two main components:
1.  **Key Material**:
    *   Identity Key Pair (Ed25519 & X25519)
    *   Signed PreKey Pair (with signature)
    *   One-Time PreKeys (all unused keys)
2.  **Message History** (Optional):
    *   Full plaintext message history
    *   Metadata (sender, timestamp, etc.)
    *   *Note: Messages are decrypted from local storage and re-encrypted with the backup password for portability.*

#### 3. Backup Flow
1.  **User Initiation**: User selects "Full Backup & Recovery" in settings.
2.  **Password Entry**: User sets a strong backup password.
3.  **Data Gathering**:
    *   `SimpleE2EEService` exports all key pairs.
    *   (Optional) `LocalMessageStore` exports all message records.
4.  **Encryption**:
    *   Salt & Nonce generated.
    *   Key derived from password via Argon2id.
    *   Payload (keys + messages) encrypted via AES-GCM.
5.  **File Generation**: JSON file containing ciphertext, salt, nonce, and metadata is saved to device.

#### 4. Restore Flow
1.  **File Selection**: User selects the `.json` backup file.
2.  **Decryption**:
    *   User enters password.
    *   Key derived using stored salt.
    *   Payload decrypted.
3.  **Import**:
    *   Keys are imported into `SimpleE2EEService` and persisted to secure storage.
    *   Messages are imported into `LocalMessageStore` and re-encrypted with the device's *new* local storage key.

### Technical Implementation
*   **Service**: `LocalKeyBackupService` handles the encryption/decryption pipeline.
*   **Store**: `LocalMessageStore` provides bulk export/import methods (`getAllMessageRecords`, `saveMessageRecord`).
*   **UI**: `LocalBackupScreen` provides the interface for creating and restoring backups.

## Security Considerations

### Current Security Model
- ✅ Perfect Forward Secrecy (PFS) via OTKs
- ✅ Post-Compromise Security via key rotation
- ✅ Authentication via Ed25519 signatures
- ✅ Confidentiality via AES-GCM

### Recovery Security Impact
- ⚠️ Breaks PFS for recovered messages
- ✅ Maintains confidentiality with password protection
- ✅ Preserves authentication via signature verification
- ⚠️ Requires trust in backup storage

### Mitigation Strategies
1. Use strong password requirements
2. Implement backup encryption verification
3. Add backup expiration policies
4. Monitor backup access patterns

## Migration Notes

### From Simple E2EE to X3DH
- Old messages encrypted with simple protocol are not decryptable with new system
- Full reset required clearing `encrypted_messages` and `profiles.identity_key`
- Multi-device support still not implemented; one account per device

### Database Migration
- Added `signed_prekeys` and `one_time_prekeys` tables
- Updated `profiles` table with new key format
- Migration scripts available in `migrations_archive/`

## Conclusion

The E2EE implementation is now fully functional with all major issues resolved. The system provides:

- Strong cryptographic guarantees
- Cross-platform compatibility
- Automatic key management
- Secure message transmission

The next phase focuses on device management to handle users with multiple active devices simultaneously.

---

**Last Updated**: February 2, 2026
**Status**: ✅ Production Ready (including key/message recovery)
**Next Priority**: Device Management (Multi-device support)