RabbitHole

The problem

The product problem is bigger than transcoding a file. A user needs to upload media, understand what is happening while it processes, play the finished video smoothly, and find useful moments later without manually scrubbing through hours of footage. RabbitHole connects those user-facing workflows to a long-running asynchronous backend without making the interface feel disconnected from the work happening behind it.

What it does

• Direct-to-S3 upload: the browser requests a presigned URL and PUTs the file straight to S3; the API never proxies the bytes.
• Async transcode: the S3 upload emits an event that fans out through EventBridge and SQS to Fargate workers running ffmpeg, producing a multi-rendition HLS ladder.
• Adaptive playback: hls.js streams the renditions from CloudFront, switching quality to match the viewer's bandwidth.
• AI auto-metadata: leave the title blank and Claude vision samples frames across the clip and writes a punchy, accuracy-guarded title, description, and tags.
• Speech-to-text + searchable captions: AWS Transcribe turns audio into caption cues; the watch page gets a searchable transcript, click-to-jump, and real WebVTT captions.
• Cross-video semantic search: a local embedding model indexes every transcript; search a phrase and it ranks the best moment in each video by meaning, then deep-links the player to that timestamp.
• Real-time status: a DynamoDB Stream drives a broadcaster Lambda that pushes Transcoding → Ready over a WebSocket, with no polling.
• Public / unlisted visibility, cost dashboard: owners publish public or unlisted; each transcode's Fargate cost is measured and surfaced.

Architecture

An event-driven pipeline that decouples the fast path (upload) from the slow path (transcode), with a serverless API and a container worker fleet, using the right tool for each job. The AI work hangs off that same backbone: synchronous vision at the worker, async event-driven speech-to-text, and an embedding model inside the API.

Highlights

• Full-stack workflow: the React interface supports upload, live processing status, HLS playback, transcript search, and semantic discovery against backend APIs and WebSocket updates.
• Deterministic scale-to-zero: a scale-up Lambda pins the autoscaling floor while work is queued; a scale-down Lambda releases it only when an idle alarm confirms the queue is empty. A stale metric can never strand a job, yet idle cost is still ~$0 (no running tasks, no NAT gateway). This replaced a naïve setup whose race could reap a freshly-queued job.
• Two AI invocation patterns: synchronous Claude vision for metadata (multi-frame, accuracy-guarded) and async, event-driven AWS Transcribe for captions. The worker fires the transcription job and returns, so transcode latency is untouched.
• Semantic search without a vector-DB bill: a small embedding model (bge-small / ONNX) is baked into the API; vectors live in DynamoDB and a brute-force cosine ranks moments. At this scale it's instant and ~free, deliberately avoiding a managed vector store (≈$700/mo) and keeping the whole thing scale-to-zero.
• Event-driven decoupling: EventBridge → SQS (with a DLQ and retries) fully separates upload from transcode: resilient to worker failure, idempotent, fan-out-ready.
• Serverless + container hybrid: Lambda runs the lightweight API; Fargate (ARM64/Graviton) runs the long-running, CPU-heavy ffmpeg. Each layer uses the right compute model.
• Private by default: CloudFront with Origin Access Control keeps the streaming bucket fully private; secrets live in SSM SecureString, never in code or state.

How it ships & runs

The operational layer is part of the project, not an afterthought; it's what turns "a deployed app" into something you can actually run and evolve.

• CI: every push/PR runs the test suites (pytest + moto for the API and caption pipeline, Vitest for the frontend) behind a coverage gate, plus an image build and terraform validate.
• CD, no static keys: a push to main assumes a least-privilege role via GitHub OIDC, builds the ARM64 images on native ARM runners, rolls out the Lambda and Fargate service, and publishes the frontend with a CDN invalidation.
• Infra through CI: Terraform state lives in a versioned, encrypted S3 bucket with native state locking; the pipeline plans on merge and applies via a deliberate manual gate (auto-applying infra is intentionally avoided).
• Observability + FinOps: a CloudWatch dashboard tracks queue depth, worker load, API latency (p95) and errors, plus a custom worker metric that surfaces $ per transcode; X-Ray traces the serverless path.
• Hardened: scoped CORS, API-gateway rate limiting, reserved admin identity, and rotated credentials before sharing.

Engineering decisions & trade-offs

Lambda API + Fargate workers

Right tool per job: serverless for the lightweight, bursty API; containers for the long-running, CPU-heavy ffmpeg transcode that would never fit Lambda's runtime and size limits.

AWS Transcribe over self-hosted Whisper

Managed and event-driven (job-complete → EventBridge → Lambda), with native word-level timestamps. It keeps the worker lean and the transcode path fast; the trade is per-minute cost versus running a model in-container.

A local embedding model, not a managed vector DB

Search embeds with a small ONNX model baked into the API and runs brute-force cosine over vectors in DynamoDB. At portfolio scale that's instant and costs nothing; OpenSearch or pgvector would be the move at a far larger corpus, but that would also reintroduce idle cost the rest of the system works hard to avoid.

Direct-to-S3 upload (presigned)

The API issues a presigned URL and the browser uploads straight to S3, so the API never proxies file bytes; this is cheaper, faster, and far friendlier to a Lambda execution model.

No NAT gateway (a documented cost trade-off)

Workers run in public subnets with a zero-ingress security group instead of private subnets behind a NAT gateway. That keeps idle cost near zero for a demo; the production trade-off is noted below.

What I'd change at scale

The demo deliberately optimizes for cost and clarity. Honest production trade-offs that remain:

• AWS Elemental MediaConvert: instead of self-managed ffmpeg, this means less operational surface and per-job billing.
• Real multi-user auth: Cognito or a user store would replace today's single-creator model with an owner field.
• Private subnets + VPC endpoints for the workers; defense-in-depth over the public-subnet demo.
• A GSI on created_at + pagination instead of a Scan for the library listing.
• OpenSearch / pgvector if the search corpus grows past brute-force; a Fargate X-Ray sidecar for worker-level tracing.
• Multi-region streaming origins with latency-based routing; signed URLs / cookies on the streaming bucket.