sojorn/sojorn_docs/features/QUIPS_VIDEO_SYSTEM.md

Quips Video System Documentation

🎬 TikTok-Level Video Recording & Editing

Version: 3.0
Status: ✅ COMPLETED
Last Updated: February 17, 2026

---

🎯 Overview

The Quips video system provides TikTok/Instagram-level video recording and editing capabilities with professional-grade features including multi-segment recording, real-time effects, and advanced processing.

🏗️ Architecture

Core Components

┌─────────────────┐    ┌─────────────────┐    ┌─────────────────┐
│ Camera Controller │    │ Video Processor │    │  UI Components   │
│                 │◄──►│                 │◄──►│                 │
│ • Camera Access  │    │ • FFmpeg Engine   │    │ • Recording UI   │
│ • Preview Stream │    │ • Frame Extraction│    │ • Effects Panel  │
│ • Recording Mgmt │    │ • Audio Mixing    │    │ • Timeline Editor│
└─────────────────┘    └─────────────────┘    └─────────────────┘
         │                       │                       │
         ▼                       ▼                       ▼
┌─────────────────┐    ┌─────────────────┐    ┌─────────────────┐
│  Device Storage │    │  Audio Library   │    │  State Manager   │
│                 │    │                 │    │                 │
│ • Segments      │    │ • Built-in Tracks │    │ • Recording State│
│ • Thumbnails    │    │ • Custom Audio   │    │ • Effects State │
│ • Temp Files     │    │ • Volume Control │    │ • Timeline State│
└─────────────────┘    └─────────┬─────────┘    └─────────────────┘

---

🎨 Features

📹 Multi-Segment Recording

• Pause/Resume: Record multiple segments with seamless transitions
• Segment Management: View, reorder, and delete individual segments
• Duration Control: 60-second maximum with real-time progress
• Auto-Save: Automatic saving of segments during recording

⚡ Speed Control

• Playback Speeds: 0.5x, 1x, 2x, 3x recording speeds
• Real-time Preview: See effects at different speeds while recording
• Speed Transitions: Smooth speed changes between segments

🎨 Filters & Effects

• Color Filters: Grayscale, Sepia, Vintage, Cold, Warm, Dramatic
• Real-time Preview: See effects applied during recording
• Effect Intensity: Adjustable strength for each filter
• Stackable Effects: Combine multiple filters

📝 Text Overlays

• Custom Text: Add text with customizable fonts and colors
• Timing Control: Set when text appears and disappears
• Positioning: Drag to position text anywhere on video
• Animation Options: Fade in/out, slide, zoom effects

🎵 Music & Audio

• Built-in Library: 6 pre-licensed music tracks
• Custom Upload: Import personal audio files
• Volume Control: Adjust audio volume with fade in/out
• Beat Sync: Optional beat synchronization for timing

✂️ Video Processing

• FFmpeg Integration: Professional-grade video processing
• Stitching: Combine multiple segments seamlessly
• Transitions: Smooth transitions between segments
• Export Options: Multiple quality and format options

---

📱 Implementation Details

Frontend Components

Enhanced Quip Recorder Screen

File: sojorn_app/lib/screens/quips/create/enhanced_quip_recorder_screen.dart

class EnhancedQuipRecorderScreen extends StatefulWidget {
  // Camera controller and preview
  late CameraController _cameraController;
  
  // Recording state
  bool _isRecording = false;
  bool _isPaused = false;
  List<File> _recordedSegments = [];
  List<Duration> _segmentDurations = [];
  
  // Effects and controls
  VideoFilter _currentFilter = VideoFilter.none;
  double _playbackSpeed = 1.0;
  bool _showTextOverlay = false;
  
  // Audio overlay
  MusicTrack? _selectedTrack;
  double _audioVolume = 0.5;
  bool _fadeIn = true;
  bool _fadeOut = true;
}

Video Stitching Service

File: sojorn_app/lib/services/video_stitching_service.dart

class VideoStitchingService {
  // Stitch multiple video segments
  static Future<File?> stitchVideos({
    required List<File> segments,
    VideoFilter? filter,
    double? playbackSpeed,
    List<TextOverlay>? textOverlays,
    MusicTrack? audioTrack,
    double audioVolume,
    bool fadeIn,
    bool fadeOut,
  });
  
  // Apply filters to video
  static Future<File?> applyFilter(
    File videoFile,
    VideoFilter filter,
  );
  
  // Extract thumbnail from video
  static Future<File?> extractThumbnail(
    File videoFile,
    Duration timePosition,
  );
}

Audio Overlay Service

File: sojorn_app/lib/services/audio_overlay_service.dart

class AudioOverlayService {
  // Mix audio with video
  static Future<File?> mixAudioWithVideo(
    File videoFile,
    File? audioFile,
    double volume,
    bool fadeIn,
    bool fadeOut,
  );
  
  // Get built-in music tracks
  static List<MusicTrack> getBuiltInTracks();
  
  // Pick custom audio file
  static Future<File?> pickAudioFile();
}

Backend Integration

Video Processing Service

File: go-backend/internal/services/video_processor.go

type VideoProcessor struct {
    ffmpegPath string
    tempDir    string
}

// Extract frames from video for moderation
func (s *VideoProcessor) ExtractFrames(
    ctx context.Context,
    videoPath string,
    frameCount int,
) ([]string, error)

// Get video duration
func (s *VideoProcessor) GetDuration(
    ctx context.Context,
    videoPath string,
) (time.Duration, error)

// Generate thumbnail
func (s *VideoProcessor) GenerateThumbnail(
    ctx context.Context,
    videoPath string,
    timePosition time.Duration,
) (string, error)

---

🎛️ User Interface

Recording Interface

• Camera Preview: Full-screen camera view with overlay controls
• Recording Controls: Record, pause, stop, and cancel buttons
• Effects Panel: Filter selection and intensity controls
• Speed Control: Playback speed adjustment
• Text Overlay: Add and position text overlays
• Audio Controls: Music selection and volume control
• Progress Indicator: Visual feedback for recording progress

Editing Interface

• Timeline View: Visual representation of recorded segments
• Segment Management: Rearrange, trim, or delete segments
• Effects Preview: Real-time preview of applied effects
• Text Editor: Edit text overlay content and timing
• Audio Mixer: Adjust audio levels and fade effects
• Export Options: Choose quality and format settings

Preview Interface

• Full Preview: Watch complete edited video before posting
• Thumbnail Selection: Choose thumbnail from any frame
• Caption Input: Add captions and descriptions
• Privacy Settings: Set visibility and audience
• Post Options: Choose posting destination

---

🔧 Technical Implementation

Camera Integration

// Initialize camera controller
Future<void> _initCamera() async {
  cameras = await availableCameras();
  
  if (cameras.isNotEmpty) {
    _cameraController = CameraController(
      cameras[0],
      ResolutionPreset.high,
      enableAudio: true,
    );
    
    await _cameraController.initialize();
  }
}

Recording Management

// Start recording segment
Future<void> _startRecording() async {
  if (_cameraController != null && !_cameraController!.value.isInitialized) {
    await _cameraController!.startVideoRecording();
    
    setState(() {
      _isRecording = true;
      _segmentStartTime = DateTime.now();
      _currentSegmentDuration = Duration.zero;
    });
    
    _progressTicker = Timer.periodic(
      const Duration(milliseconds: 100),
      (timer) => _updateProgress(),
    );
  }
}

// Stop recording segment
Future<void> _stopRecording() async {
  if (_isRecording) {
    _progressTicker?.cancel();
    
    final videoFile = await _cameraController!.stopVideoRecording();
    
    if (videoFile != null) {
      // Add segment if it has content
      if (_currentSegmentDuration.inMilliseconds > 500) {
        _recordedSegments.add(videoFile);
        _segmentDurations.add(_currentSegmentDuration);
      }
      
      setState(() => _isRecording = false);
    }
  }
}

Video Processing

// Process recorded video
Future<void> _processVideo() async {
  if (_recordedSegments.isEmpty) return;
  
  try {
    // Stitch segments together
    final stitchedVideo = await VideoStitchingService.stitchVideos(
      segments: _recordedSegments,
      filter: _currentFilter,
      playbackSpeed: _playbackSpeed,
      textOverlays: _textOverlays,
      audioTrack: _selectedTrack,
      audioVolume: _audioVolume,
      fadeIn: _fadeIn,
      fadeOut: _fadeOut,
    );
    
    if (stitchedVideo != null) {
      // Generate thumbnail
      final thumbnail = await VideoStitchingService.extractThumbnail(
        stitchedVideo,
        Duration(seconds: 1),
      );
      
      // Navigate to post creation
      Navigator.of(context).push(
        MaterialPageRoute(
          builder: (_) => CreatePostScreen(
            videoFile: stitchedVideo,
            thumbnailFile: thumbnail,
          ),
        ),
      );
    }
  } catch (e) {
    _showError('Video processing failed: $e');
  }
}

---

📊 Performance Optimization

Memory Management

• Segment Storage: Efficient storage of video segments
• Preview Caching: Cached preview frames for smooth UI
• Background Processing: Async video processing to avoid UI blocking
• Memory Cleanup: Automatic cleanup of temporary files

Recording Performance

• Camera Optimization: Optimized camera settings for smooth recording
• Frame Rate Control: Consistent 30fps recording
• Bitrate Management: Adaptive bitrate based on content
• Storage Optimization: Compressed video storage

Processing Speed

• FFmpeg Optimization: Optimized FFmpeg parameters for fast processing
• Parallel Processing: Multiple effects applied simultaneously
• Hardware Acceleration: GPU-accelerated video processing when available
• Progressive Loading: Incremental video loading and processing

---

🔒 Security & Privacy

Content Moderation

• Real-time Analysis: Video frames analyzed during upload
• AI Integration: OpenAI Vision API for content safety
• User Reporting: Built-in reporting for inappropriate content
• Automated Flagging: Automatic detection of policy violations

Data Protection

• Local Storage: Video segments stored locally during recording
• Secure Upload: Encrypted upload to Cloudflare R2
• Privacy Controls: User-controlled visibility settings
• Data Retention: Automatic cleanup of temporary files

User Safety

• Content Guidelines: Clear community guidelines for video content
• Reporting Tools: Easy-to-use reporting for inappropriate content
• Moderation Queue: Human review of flagged content
• Appeal Process: Fair appeal system for content decisions

---

📱 Platform Support

Android

• Camera Access: Full camera control with permissions
• Storage: Local file system with efficient management
• Performance: Optimized for mobile devices
• Codecs: H.264/AVC for video, AAC for audio

Web

• WebRTC: Browser-based camera access
• MediaRecorder API: Standard web recording capabilities
• File Handling: Browser file system integration
• Performance: Web-optimized video processing

iOS (Planned)

• AVFoundation: Native iOS camera integration
• Core Media: Professional video processing
• Photos Library: Integration with device photo library
• Performance: Hardware-accelerated video processing

---

🧪 Testing

Unit Tests

testWidgets('Video recording functionality', (WidgetTester tester) async {
  await tester.pumpWidget(EnhancedQuipRecorderScreen());
  
  // Test recording controls
  await tester.tap(find.byType(ElevatedButton));
  expect(find.byIcon(Icons.stop), findsOneWidget);
  
  // Test pause/resume
  await tester.tap(find.byIcon(Icons.pause));
  expect(find.byIcon(Icons.play_arrow), findsOneWidget);
  
  // Test effects
  await tester.tap(find.byType(FilterButton));
  expect(find.byType(FilterSelector), findsOneWidget);
});

Integration Tests

test('Complete video recording workflow', () async {
  // Test full recording -> editing -> posting flow
  final recorder = EnhancedQuipRecorderScreen();
  final segments = await recorder.recordVideo();
  final editedVideo = await recorder.editVideo(segments);
  final post = await recorder.createPost(editedVideo);
  
  expect(post.videoUrl, isNotNull);
  expect(post.thumbnailUrl, isNotNull);
});

Performance Tests

test('Recording performance benchmarks', () async {
  final stopwatch = Stopwatch()..start();
  
  await recordVideo(duration: Duration(seconds: 30));
  
  stopwatch.stop();
  expect(stopwatch.elapsedMilliseconds, lessThan(1000));
});

---

🔧 Configuration

FFmpeg Setup

# Install FFmpeg for video processing
sudo apt update
sudo apt install ffmpeg

# Verify installation
ffmpeg -version

Flutter Dependencies

dependencies:
  camera: ^0.10.5+2
  ffmpeg_kit_flutter: ^6.0.0
  video_player: ^2.8.1
  image_picker: ^1.0.4
  file_picker: ^6.1.1

Platform Configuration

// Video recording configuration
class VideoConfig {
  static const int maxDuration = 60; // seconds
  static const int maxSegments = 10;
  static const int maxFileSize = 100 * 1024 * 1024; // 100MB
  static const String defaultQuality = 'high';
  static const List<String> supportedFormats = ['mp4', 'mov', 'avi'];
}

---

📚 API Reference

Video Processing Endpoints

// POST /api/video/process
type VideoProcessRequest struct {
    Segments    []string `json:"segments"`
    Filter      string  `json:"filter"`
    Speed       float64 `json:"speed"`
    TextOverlay []TextOverlay `json:"textOverlays"`
    AudioTrack   *AudioTrack `json:"audioTrack"`
    Volume      float64 `json:"volume"`
    FadeIn      bool    `json:"fadeIn"`
    FadeOut     bool    `json:"fadeOut"`
}

// GET /api/video/thumbnail
type ThumbnailRequest struct {
    VideoPath string `json:"videoPath"`
    TimePosition int64 `json:"timePosition"`
}

Audio Library Endpoints

// GET /api/audio/tracks
type AudioTrackResponse struct {
    Tracks []MusicTrack `json:"tracks"`
    Total  int         `json:"total"`
}

// POST /api/audio/upload
type AudioUploadRequest struct {
    Name     string `json:"name"`
    File     []byte `json:"file"`
    Duration int64  `json:"duration"`
}

---

🚀 Deployment

Environment Variables

# FFmpeg configuration
FFMPEG_PATH=/usr/bin/ffmpeg
VIDEO_TEMP_DIR=/tmp/sojorn_videos
VIDEO_MAX_SIZE=104857600  # 100MB

# Video processing
VIDEO_QUALITY=high
VIDEO_FORMAT=mp4
VIDEO_FPS=30
VIDEO_BITRATE=2000000

Health Checks

// Video processing health check
func (s *VideoService) HealthCheck() HealthStatus {
    // Check FFmpeg availability
    _, err := exec.LookPath("ffmpeg")
    if err != nil {
        return HealthStatus{
            Status: "unhealthy",
            Message: "FFmpeg not available",
        }
    }
    
    // Check temp directory
    if _, err := os.Stat(s.tempDir); os.IsNotExist(err) {
        return HealthStatus{
            Status: "degraded",
            Message: "Temp directory not accessible",
        }
    }
    
    return HealthStatus{
        Status: "healthy",
        Message: "Video processing ready",
    }
}

---

📈 Troubleshooting

Common Issues

Camera Not Working

// Check camera permissions
if (!await Permission.camera.isGranted) {
  await Permission.camera.request();
}

// Check camera availability
final cameras = await availableCameras();
if (cameras.isEmpty) {
  throw Exception('No camera available');
}

Video Processing Errors

try {
  final result = await VideoStitchingService.stitchVideos(
    segments: segments,
    filter: filter,
  );
} on VideoProcessingException catch (e) {
  print('Video processing failed: $e');
  // Show user-friendly error message
  _showError('Video processing failed. Please try again.');
}

Storage Issues

// Check available storage
final storageInfo = await getStorageInfo();
if (storageInfo.availableSpace < 100 * 1024 * 1024) {
  _showError('Insufficient storage space');
  return;
}

---

📝 Future Enhancements

Version 3.1 (Planned)

• Real-time Effects: Apply effects during recording
• Advanced Transitions: More transition options between segments
• Collaborative Editing: Multiple users editing same video
• Voice Recording: Add voice-over capabilities

Version 4.0 (Long-term)

• AI Video Enhancement: AI-powered video enhancement
• 3D Effects: 3D video effects and animations
• Live Streaming: Real-time video streaming capabilities
• Advanced Analytics: Video performance metrics and insights

---

📞 Support

Documentation

• User Guide: Complete user manual for video features
• Developer Guide: Technical implementation details
• API Reference: Complete API documentation
• Troubleshooting: Common issues and solutions

Community

• Discord: Video development discussion channel
• GitHub: Issue tracking and feature requests
• Documentation: Regular updates and improvements

---

🎬 The Quips video system provides professional-grade video recording and editing capabilities, enabling users to create engaging short-form video content with TikTok-level features and professional polish.