baka Codes

Announcing Vortex v1.0.0: Cloud-Native Distributed Video Infrastructure Engine I built Vortex, an open-source distributed video transcoding and streaming platform written in pure Go. It ingests raw media streams, coordinates workers via asynchronous message brokers, segments video into multi-bitrate HLS playlists, tracks real-time encoding progress, and executes durable post-processing workflows. Key Features - Multi-Protocol Ingestion: Streaming HTTP multipart upload (32KB constant memory footprint) and high-throughput gRPC client streaming via Protocol Buffers with server reflection. - Distributed Transcoding: Parallel FFmpeg processing pipeline chunking video into 360p, 720p, and 1080p Apple HLS master playlists and transport segments (.ts). - Real-Time Progress: Live WebSocket broadcast streaming millisecond FFmpeg progress directly to web players via Redis Pub/Sub. - Event-Driven Architecture: NATS JetStream 2.10 at-least-once message delivery with durable worker consumer groups. - Durable Workflows: Inngest step-based execution engine with automatic checkpointing for poster extraction, animated GIF preview generation, and webhook delivery. - Defense & Rate Limiting: Caddy edge reverse proxy with unbuffered 500MB payload streaming paired with an atomic Redis sliding-window token bucket rate limiter. - Full-Stack Observability: OpenTelemetry distributed tracing with W3C context propagation across NATS, Prometheus RED metrics, and pre-provisioned Grafana monitoring cockpits. - Cloud-Native Orchestration: Parameterized Helm 3 charts on Kubernetes with non-root multi-stage containers and CPU-driven Horizontal Pod Autoscaling (HPA). Core Tech Stack Go, PostgreSQL (pgx/v5), Redis 7, NATS JetStream, MinIO S3, FFmpeg, Gorilla WebSockets, gRPC, Inngest, OpenTelemetry, Prometheus, Tempo, Caddy, Kubernetes, Helm 3. GitHub Repository: https://github.com/ruhamabek/vortex License: MIT