Fogsight: The End of the Black Box Video

How an open-source agent uses Gemini Pro and GSAP to replace diffusion models with perfectly editable, code-generated animations.

6 min read • View on GitHub • More from fogsightai

An illustration showing a massive industrial printing press churning out a solid block of dark metal, while nearby, a delicate pair of jeweler's tweezers assembles a complex clockwork mechanism. This contrasts the monolithic nature of diffusion video models with the granular, code-based approach of Fogsight. — Diffusion models generate uneditable pixels. Fogsight generates the underlying code that renders the animation.

Key Takeaways

Fogsight flips the AI video paradigm by functioning as a code-generating agent that outputs HTML, CSS, and GSAP JavaScript instead of flat MP4 files.
The project relies on the massive context window of Gemini 2.5 Pro to stream hundreds of lines of functional, unbroken UI logic in a single pass.
By generating source code, Fogsight enables a 'Language User Interface' where animations can be iteratively refined and edited via natural language commands.
To manage the security risks of executing AI-generated code locally, the system relies on heavily restricted HTML iframes.

The Illusion of Video

Most AI video generators treat animation as a problem of pixel prediction. Tools like Sora or Kling use diffusion models to hallucinate each frame, resulting in a heavy, flat file. If you want to change the color of a specific element or adjust the timing of a transition, you have to write a new prompt and roll the dice again.

Fogsight takes a radically different approach. It is not a video generator; it is a specialized coding agent disguised as an animation tool. Instead of rendering pixels, it writes bespoke, self-contained web applications and renders them live in the browser.

Feature	Diffusion Video Models	Fogsight (Code Agent)
Underlying Mechanism	Pixel Hallucination / Diffusion	Logic Generation / LLM Coding
Output Format	Flat file (.mp4)	Web App (HTML, CSS, JS, SVG)
Editability	Prompt-and-pray	Granular, component-level editing
File Size	Gigabytes	Kilobytes
Best For	Photorealism, B-roll	Educational diagrams, UI mockups

The "Gemini-First" Engine

The architecture behind Fogsight is a classic "Thin Backend, Rich Frontend" pattern, heavily reliant on a single, powerful Large Language Model. The FastAPI backend, specifically app.py, serves as the orchestrator.

The system prompt is the core of the project. It instructs the LLM to output a single-file HTML bundle, explicitly requiring 2k resolution and bilingual subtitles. To achieve this, Fogsight heavily prioritizes Gemini 2.5 Pro. The model's massive context window is necessary to generate 500+ lines of functional GSAP (GreenSock Animation Platform) JavaScript in a single, unbroken stream without losing track of the animation's logic.

Fogsight extracts and renders code from a raw LLM stream in real-time.

The Language User Interface (LUI)

Because the output is code, the animation is inherently editable. Fogsight's frontend, managed by script.js, leverages this to create what it calls a Language User Interface (LUI).

An illustration of a classic wooden film director's chair facing a stage. An ink pen is writing stage directions directly onto a continuous paper scroll that feeds into the mechanical actors' scripts. This represents the Language User Interface replacing traditional timeline-based editing. — Iterative editing is handled through conversational commands rather than a traditional timeline interface.

The system maintains a conversation history. If a user wants to change a specific element—say, "make the background darker"—the request is sent back to the LLM along with the existing code context. The LLM then streams the updated JavaScript, which the frontend parses and re-injects into the viewer.

通过与 AI 的多轮对话，您可以对动画进行精准调优和迭代，直至达到您心中最理想的艺术效果。

fogsightai/fogsight, Project Repository · Fogsight (雾象) English | 中文

The Sandbox Reality

Executing AI-generated code locally presents a significant security challenge. An LLM hallucination or a malicious prompt injection could theoretically result in harmful JavaScript execution.

An illustration of a tiny, perfectly rendered terrarium sealed inside a thick glass bell jar. Inside the jar, a miniature violent thunderstorm is raging, while the outside environment—a pristine wooden desk with neatly stacked papers—remains completely unaffected. This visualizes the security sandbox containing the AI-generated code. — The generated web applications are isolated within a restricted iframe to prevent access to the parent page's DOM or cookies.

Fogsight mitigates this risk at the presentation layer. In templates/index.html, the rendering environment is a heavily restricted <iframe> utilizing the sandbox="allow-scripts allow-same-origin" attribute. This ensures the output remains isolated.

The Future of Educational Content

The true value of Fogsight lies in its application for abstract concepts. Photorealistic video generators struggle to accurately portray mathematical formulas or sorting algorithms. By generating logic rather than pixels, Fogsight provides a scalable, infinitely editable format for science communicators and educators.

该工具主要面向教育工作者、科普内容创作者和产品经理等需要快速将复杂概念视觉化的用户。

fogsight.aigcsoft.site, Tool Description Website · Fogsight：输入文本即可生成动画的AI引擎

It represents a shift toward transparent AI media, where the underlying source code of an animation is always available for human-in-the-loop editing.