atlascineesp: The Small Python Pipeline Building a Cinema Atlas
How a two-stage TMDB harvester, a few CSVs, and Power BI turn film metadata into a research tool for comparing countries, crews, and trends over time.
- atlascineesp treats film metadata as a research instrument, not a byproduct, by aiming at a long-lived cinema atlas instead of a one-off scrape.
- Its architecture separates discovery from enrichment, so the pipeline can first find films and then deepen only the records that matter.
- The repo's real differentiator is its attention to underused crew roles and director gender, which makes the dataset more useful for film-history questions.
- A config-driven country switch and Power BI output turn a lightweight local stack into a reusable template for comparative cinema studies.
The repo is really a cinema atlas, not a scraper
Most movie scraping projects stop at titles, cast lists, and maybe a poster URL. atlascineesp is aiming at something more useful: a longitudinal cinema atlas that can support questions about budgets, genres, crew roles, and gender patterns in directing. That is a different class of tool.
The repo's ambition is bigger than its stack. It uses Python, CSVs, and Power BI, but the target is a research workflow: discover films, enrich them, preserve them, and make them queryable over time. The result is small-data engineering with an actual editorial point of view.
Discovery first, enrichment second
The repo is split into two clear moves. recuperaPeliculas.py discovers films through TMDB's discover endpoint, paginates through results, and writes a seed CSV with basic records. datasetEnriquecido.py then takes those IDs and performs the deeper harvest, calling the movie detail endpoint to pull a much richer record for each film.
# Discovery phase
# 1. Query TMDB discover endpoint by country
# 2. Paginate through results
# 3. Write seed rows to movie_data.csv
# Enrichment phase
# 1. Read seed IDs
# 2. Fetch detailed movie data
# 3. Extract crew roles and metadata
# 4. Save enriched CSV for analysisThat separation matters because it keeps the pipeline legible. One script answers "what exists?" and the other answers "what else do we need to know?" The design is simple, but the question model is strong.
The clever part is the delta merge
The most interesting technical move is the incremental refresh. The code uses a pandas outer merge and then filters for right_only rows, which means the enrichment step only runs for films that are in the seed list but not already present in the enriched dataset. That is a small trick with a big payoff: fewer redundant API calls, less churn, and a dataset that can grow without starting from zero every time.
dataset_nuevas = pd.merge(
dataset_inicial,
lista_con_id,
how='outer',
on='id',
indicator=True
)
dataset_nuevas = dataset_nuevas[dataset_nuevas['_merge'] == 'right_only']This is the sort of code that looks ordinary until you realize what it enables. The pipeline behaves more like a living archive than a throwaway scraper.
Why crew metadata changes the shape of the dataset
A basic movie dataset tells you what was released. This one tries to tell you how a film was made. By extracting crew roles like cinematography, editing, screenplay, and direction, the repo moves beyond cast-centric metadata and starts to capture the people who shape form.
| Basic movie scraper | atlascineesp |
|---|---|
| Stops at title and cast. | Tracks crew roles that matter to film form. |
| Treats the record as a flat row. | Builds a research object for historical comparison. |
| Reprocesses everything on each run. | Uses a delta merge to enrich only missing films. |
| Usually ends in a CSV export. | Ends in CSV plus Power BI for analysis. |
| Hard to compare across regions. | Uses configuration to shift country or territory focus. |
That difference is editorial, not just technical. If you care about cinema history, the editor and cinematographer are not edge cases. They are often the people who most clearly shape the work you're trying to study.
Config turns one country into many
The repository is built to switch geographies without rewriting the core logic. A configuration file controls the target country, which means the same pipeline can be reused for Spanish cinema, then adapted for Chinese-language territories like China, Hong Kong, Taiwan, and Macau. That is what makes it feel like a template for comparative cultural analysis rather than a single-purpose script.
There is still a handmade feel to it. Some file paths appear to be updated manually, which is normal in a personal research project. The point is not enterprise abstraction. The point is that the pipeline is flexible enough to support new questions without becoming a framework.
A personal ETL with BI ambitions
The presence of a Power BI dashboard file is the final clue. This repo is not just collecting data for its own sake. It is building a local ETL system that ends in analysis, which is why the semicolon-separated CSVs and the historical context files make sense together.
That combination keeps the project grounded. It does not need Spark, orchestration, or a warehouse to be serious. It needs a clear question, a durable schema, and a repeatable way to grow the atlas over time.
