Voice & Multimodal AI

OpenBMB’s tokenizer-free speech system turns text, voice descriptions, and short reference clips into continuous, high-fidelity speech.

A monorepo for embodied companions, where one TypeScript core can wear different bodies, speak with a cloned voice, and step into worlds like Minecraft.

How Google’s cross-platform pipeline turns video, audio, and sensor data into low-latency perception by deciding what to process, what to drop, and what to pass through.

How a hybrid LLM-diffusion architecture squeezed 90 minutes of multi-speaker audio into a single pass, prompting a corporate redaction.

A deep dive into how Fish Speech combines dual-autoregressive transformers, repetition-aware sampling, and codec-level decoding to make open-source TTS sound controlled, expressive, and fast.

A deep dive into the Rust core, CEF shell, local memory vault, and the fake-camera trick that lets an assistant show up inside your workflow instead of beside it.

An open-source desktop app for voice cloning and speech generation that bundles the model, the backend, the editor, and the effects chain into one private workflow.

A deep dive into the Rust-and-SQLite system that records everything you see and hear, then lets AI agents search, replay, and act on it without sending your life to the cloud.

An open-source voice system that generates multilingual speech in continuous latent space, blends voice design with cloning, and aims for studio-quality output without the usual codebook bottleneck.

A modular Python pipeline that swaps in local STT, LLM, and TTS backends, mimics the OpenAI Realtime API, and solves the hardest part of voice UX: stopping cleanly when the user starts talking.

A deep dive into the agentic pipeline, the camera tree, and the structured shot logic that try to solve the hardest problem in video generation: staying coherent from scene to scene.

How full-page multimodal decoding is replacing twenty years of fragile document pipelines to finally solve the messy reality of handwriting and math.

How Software Mansion turned React Native’s biggest weakness into a 120fps superpower through the magic of UI-thread Worklets.

How a single Go binary reconciles thirty years of incompatible protocols to create a "Unified Field Theory" for online conversation.

A native Swift app, a thin Cloudflare proxy, and a tiny coordinate language turn Claude into a spatial tutor that follows your cursor instead of hiding in a sidebar.

A non-invasive chat assistant that combines Accessibility, OCR, and split-model reasoning to draft replies without rooting, hooking, or auto-sending.

A Codex Skill that uses a recurring black character, low-tech metaphors, and strict style rules to generate consistent hand-drawn editorial art for Chinese technical writing.

How a 100M-parameter TTS stack uses audio tokenization, ONNX, and streaming decode to make high-fidelity voice generation feel local, fast, and practical.

A minimalist local-first tool that captures mic and system audio separately, writes crash-safe CAF files, and transcribes on-device without a cloud meeting bot.

An open-source runtime that keeps agents talking while they work, separates real-time presence from task execution, and wraps ACP-compatible backends in a persistent voice interface.

A tiny ESP32 companion turns Claude Desktop state into motion, mood, and physical approvals, then ships custom assets to the device over BLE with a drag-and-drop workflow.

A deep look at how this skill uses seed templates, layout recipes, and rendered-image validation to turn raw content into polished Xiaohongshu and WeChat graphics without letting the model freestyle.

How an open-source agent uses Gemini Pro and GSAP to replace diffusion models with perfectly editable, code-generated animations.

How an open-source Python CLI uses multimodal AI and overlapping video chunking to replace timeline scrubbing with natural language search.

A local-first tool that turns raw video into scene-aware keyframes, deduplicated grids, and a manifest an LLM can actually use.

How a TypeScript desktop client tames the OpenClaw runtime to bring private, autonomous agents to everyday messaging apps.

How the Model Context Protocol standardized interactive, sandboxed UIs to turn AI clients into fully-fledged operating systems.

A Rust-powered takeover of the Paper Pro that erases the interface, turns handwriting into a ritual, and uses an LLM as the voice behind the page.

A deep dive into how `isair/jarvis` turns ambient speech, rolling transcript windows, and layered memory into a privacy-first assistant that can join a conversation without being reintroduced.

A look at how Meta turns ESP32 boards and Raspberry Pis into voice-first gadgets, with one firmware stack, one pairing flow, and one cloud brain behind the scenes.

How a Rust-powered Electron core bypasses screen-sharing detection to provide real-time, local-first interview intelligence.

A deep dive into the local-first AI tutor that turns real-time conversation, spaced repetition, and your own API key into a private path to fluency.

It generates continuous motion, cleans the frames, chooses the right delivery format, and maps scroll, mouse, drag, touch, or device orientation onto the same sequence.

How Vercel built a unified SDK that translates React-like components and streaming AI into native Slack, Teams, and Discord apps.

Behind the scenes, one upload becomes a staged job, a reviewed cutout, a modeled preview, and a searchable library that never leaves your machine.

A Vercel Labs template that uses Gemini, tools, and typed React components to turn chat into a task flow instead of a transcript.

A lightweight Python app that calibrates to your neutral face, scores expressions relative to that baseline, and pushes live meme overlays into Zoom, Meet, and Discord as a virtual camera.

A cross-platform Electron app that captures system output, lip-syncs a 3D avatar, and gives agents a body they can actually use.

A FastAPI and React reference repo where state, widgets, hidden context, and client effects do the work that plain prompting cannot.

A privacy-first audio workspace that swaps cloud APIs for local models, then adds diarization, notebook workflows, and an unusually structured AI-agent build system.

An AI-native media engine where MulmoScript becomes the source of truth, the browser becomes the renderer, and the same narrative can ship in multiple formats without rewriting the story.

A look at how this Tsinghua-built repo uses time-frequency interleaving and gain extraction to chase strong separation without the usual model bloat.

Inside the widget diff engine, typed protocol, and persistence layer that turn a Python backend into a living chat interface.

A SwiftUI vision wrapper built around a proxy-first trust boundary, automatic camera prompting, and a deliberately boring backend that makes mobile AI safer to ship.

A synthetic audio stack that treats motion as the signal, not metadata, then turns that motion into a dataset and benchmark for speech enhancement and separation.

How a lightweight codebook and a clever loss function gave Large Multimodal Models spatial agency without the heavy overhead of Segment Anything.

A fast, hardened Telegram Mini App for the Hermes agent. It does more than chat: it manages sessions, system status, cron jobs, and fork maintenance without losing the terminal feel.

A sliding-window VLM pipeline shows that, for streaming video, the smartest move may be to stop chasing memory and let the model focus on the last few frames.