CognifyAI Turns Videos and PDFs Into a Study Engine

A FastAPI and Next.js project that extracts content, builds embeddings, generates flashcards and quizzes, and layers RAG chat on top of the same learning core.

8 min read • View on GitHub • More from Adittii72

A wide black-ink illustration of a content machine with two inputs feeding one mechanism. A strip of video frames and a stack of documents enter on the left, and flashcards, quiz cards, and a chat window come out on the right, showing how one source becomes several study formats.
CognifyAI is built around transformation, not just conversation. It turns passive media into a pipeline of learning outputs.
Key Takeaways

CognifyAI starts with a simple promise in its README: it turns YouTube videos and PDFs into interactive learning materials. That sounds familiar until you notice the shape of the output. This is not a chatbot with a file upload. It is a system for converting passive media into study assets.

🎓 AI-Powered Learning Assistant Transform YouTube videos and PDFs into interactive learning materials with AI-powered flashcards, quizzes, and contextual chat.

Project README, Repository documentation · Adittii72/CognifyAI README

From passive media to active study material

That distinction matters because it changes what the product is optimizing for. A generic docs chatbot aims to answer questions. CognifyAI tries to produce learning behavior: first extract the material, then reshape it into quizzes and flashcards, then let the learner interrogate the same source through chat.

The repo’s core idea is closer to curriculum assembly than conversation design. One input can feed multiple outputs, and each output serves a different part of the study loop. That is the thesis that makes the project interesting.

Generic chat-with-docs appCognifyAI
Upload a file, ask questions, get answersIngest a source, extract structure, and generate flashcards, quizzes, and chat context
Optimized mainly for lookupOptimized for learning workflows
Usually one output modeMultiple output modes from the same source
Chat is the productRetrieval is the backbone of the product

The orchestrator behind the app

The backend entry point, backend/main.py, acts as the contract layer between the interface and the learning pipeline. It uses Pydantic models to validate requests, coordinates extraction, and returns a content_id that becomes the session handle for later chat and generation calls.

That is a quiet but important design choice. It keeps the app stateful at the product level while leaving the service layer modular. The frontend does not need to know how extraction or vector storage works. It only needs to keep track of which content the learner is working on.

Create an interactive-style SVG system diagram for a learning pipeline. Show one input on the left

The real core is retrieval, not chat

The strongest technical idea in CognifyAI lives in vector_service.py and ai_service.py. Text is chunked into overlapping windows, embedded, stored, and then retrieved when the app needs to generate a quiz, a flashcard set, or a contextual answer. Chat is just one consumer of the same retrieval layer.

A close-up black-ink illustration of a retrieval mechanism. Text chunks are sliced into overlapping strips, dropped into a vector grid, and then pulled back out by a magnifying-glass query, with a small fallback rail bypassing a broken gate.
The technical surprise is not the model choice. It is the way retrieval is made reusable, and how the app keeps moving when one dependency fails.

The implementation details are practical rather than glamorous. The code uses a Gemini embedding model when it can, but it also includes a hash-based fallback so the whole app does not collapse if the API blips. That is not a perfect substitute for real embeddings, but it is a real-world reliability move.

Why the fallback matters

This is one of those choices that tells you the author has built with production failure in mind. If the embedding call fails, the app degrades instead of dying. That matters more in learning products than in many other apps, because the user experience depends on continuity across sessions and repeated study loops.

# Conceptual view of the retrieval flow
chunks = split_text(content, chunk_size=1000, overlap=200)
embeddings = get_embedding(chunks)  # Gemini first, hash fallback if needed
store(content_id, chunks, embeddings)

# Later
matches = search_similar_chunks(query_embedding, content_id)
answer = generate_response(query, matches)

YouTube and PDF are treated differently on purpose

The ingestion layer reflects a useful bias. YouTube content is handled through subtitle extraction with yt-dlp and VTT parsing, which is much safer than trying to process full video. PDFs are handled with PyPDF2, which fits the linear, page-based nature of documents.

Source typeExtraction pathTradeoff
YouTubeSubtitle retrieval via yt-dlp and VTT parsingFast and practical, but dependent on transcript availability
PDFPage text extraction via PyPDF2Straightforward, but limited by PDF text quality
Unified resultPlain text for chunking and embeddingKeeps downstream logic consistent

The important thing is that the system does not pretend these inputs are the same. It normalizes them into plain text, but it respects the reliability differences between them. That makes the later retrieval layer simpler and more predictable.

Two storage modes make the repo easier to use

CognifyAI has a nice DX trick: production storage lives in Supabase with pgvector, while simple_storage.py gives the project a local path for testing. That means you can work on the app without immediately wiring up a full database stack.

This is the sort of decision that helps a repo move from promising to usable. It reduces setup friction, which matters if the project is meant to be explored, extended, or demoed by someone new.

ModePurposeWhat it signals
Supabase plus pgvectorProduction-ready vector storageDesigned for real retrieval workloads
simple_storage.pyLocal testing and developmentDesigned for fast onboarding and low friction

Structured output is the other half of the story

The prompt layer in ai_service.py pushes hard for structured JSON responses, especially for flashcards and quizzes. That sounds mundane until you remember how often LLM integrations fail on formatting, wrapper text, or markdown fences.

The repo’s parsing logic acknowledges that reality instead of ignoring it. Good AI app design is often less about prompting genius and more about surviving model output that is almost correct.

# Conceptual prompt shape
"Return ONLY a JSON array of flashcards."

# Parser tolerance
# - strip markdown fences
# - parse the payload
# - recover clean JSON for the frontend

Where CognifyAI fits in the learning-app landscape

Against a generic chat-with-docs product, CognifyAI is more opinionated. It does not stop at retrieval. It tries to package retrieval into study behavior, which makes it closer to a learning workflow engine than a general-purpose assistant.

DimensionGeneric docs chatCognifyAI
Primary jobAnswer questionsBuild study materials and answer questions
Learning loopThinExplicit
System shapeSingle interfacePipeline plus interfaces
DifferentiatorConvenienceTransformation

That also explains the repo’s maturity read. It looks like a serious prototype, with modular services, clear boundaries, and production-shaped decisions. It also keeps a few rough edges, like in-memory fallback paths and synchronous pieces wrapped inside async flows, which is exactly what a working early-stage system often looks like.

What the repo signals about maturity

The strongest signal here is not polish for its own sake. It is the structure. The backend is split into services, the frontend is separated cleanly, and the data path is legible from ingestion to generation. That kind of clarity makes the project easier to extend without turning it into a ball of prompts and callbacks.

CognifyAI is interesting because it treats retrieval not as a chat feature, but as the backbone of a study system that turns passive content into active learning assets.