The Data Foundry

Built by Data with Pranjal

Public Beta

The Data Foundry is improving every week. If something feels unclear, broken, or missing, tell us and we will use it to improve the platform.

Core Practice Path

Know what to practise next without losing the full library.

Follow 30 reviewed exercises in a deliberate order, or open the complete SQL, Python, PySpark, Airflow, AWS, scenario, and system-design libraries whenever you need more repetition. The core is a recommendation, not an access limit.

Progress comes from completed exercises. Completing this path records practice evidence; it does not guarantee an interview or job outcome.

Core progress

0/30

reviewed exercises completed

0%

Start: Top salary bands per department

01

Core for direction

Use the reviewed sequence when you do not know what should come next.

02

Library for depth

Open every track's full problem collection for repetition and interview drilling.

03

Evidence for progress

Stages complete from actual exercise results, never from a manual checkbox.

Your learning sequence

Seven stages, thirty core exercises

The curated exercises establish a dependable foundation. The secondary action in every stage keeps the entire practice library available from the same place.

Stage 1Current stage

0/8 core exercises

SQL correctness

Build reliable grain, joins, windows, deduplication, change detection, and late-data reasoning.

Top salary bands per department

Partitioned ranking and tie handling

Beginner · 20 min

Daily revenue running total

Aggregate before applying a window

Beginner · 18 min

Month-over-month revenue change

Time-grain aggregation and LAG

Beginner · 20 min

Detect duplicate business keys

Unlock Premium to open this exercise.

Sessionize events by inactivity gap

Unlock Premium to open this exercise.

Keep the latest row per business key

Unlock Premium to open this exercise.

Detect SCD Type 2 changes

Unlock Premium to open this exercise.

Find late facts in the wrong partition

Unlock Premium to open this exercise.

Ready to move on when you can

Return the correct business grain without duplicates
Handle ties, NULLs, and deterministic row selection
Explain why the query stays correct on edge cases

Continue core path Explore all SQL problems

Stage 2

0/5 core exercises

Python for pipelines

Use Python for validation, idempotency, reconciliation, deterministic selection, and CDC handling.

Deduplicate replayed events

Idempotent event handling

Beginner · 12 min

Build a required-field reject list

Data-quality validation and quarantine

Beginner · 12 min

Keep the latest update per key

Unlock Premium to open this exercise.

Reconcile source and warehouse keys

Unlock Premium to open this exercise.

Apply CDC events to current state

Unlock Premium to open this exercise.

Ready to move on when you can

Write readable functions with explicit input and output contracts
Test empty, duplicate, malformed, and missing-value cases
Preserve deterministic behavior when inputs are replayed

Continue core path Explore all Python problems

Stage 3

0/5 core exercises

PySpark production

Practise deterministic DataFrame transformations, schema evolution, timezones, and join strategy.

Keep the latest customer version

Unlock Premium to open this exercise.

Union evolving schemas safely

Unlock Premium to open this exercise.

Build the correct local business date

Unlock Premium to open this exercise.

Preserve a new field after schema drift

Unlock Premium to open this exercise.

Broadcast the correct join side

Unlock Premium to open this exercise.

Ready to move on when you can

Choose deterministic windows and schema-safe transformations
Reason about timezones, joins, shuffle, and partition behavior
Explain the production consequence of the original bug

Continue core path Explore all PySpark problems

Stage 4

0/4 core exercises

Airflow and operations

Recover safely from backfill, retry, observability, and orchestration failures.

Prevent a backfill from doubling revenue

Unlock Premium to open this exercise.

Recover from an API rate-limit storm

Unlock Premium to open this exercise.

Restore missing worker logs

Unlock Premium to open this exercise.

Replace brittle cron coupling

Unlock Premium to open this exercise.

Ready to move on when you can

Separate scheduler, worker, dependency, and downstream failures
Design idempotent recovery and bounded retries
Name the evidence and monitoring needed to verify recovery

Continue core path Explore all Airflow problems

Stage 5

0/2 core exercises

AWS judgment

Make cloud decisions from workload evidence, delivery guarantees, cost, and operating constraints.

Make duplicate S3 events harmless

Unlock Premium to open this exercise.

Choose Glue or EMR for a nightly workload

Unlock Premium to open this exercise.

Ready to move on when you can

Start with workload, scale, latency, and failure constraints
Compare the chosen service with its closest alternative
Include security, cost, idempotency, and observability

Continue core path Explore all AWS problems

Stage 6

0/2 core exercises

Broken-pipeline scenarios

Combine technical diagnosis, business impact, safe recovery, and prevention in realistic incidents.

Recover revenue after a late source arrival

Late data, reconciliation and safe recovery

Intermediate · 25 min

Diagnose a revenue drop after a status change

Unlock Premium to open this exercise.

Ready to move on when you can

Use the supplied evidence before proposing a fix
State the root cause and affected business outcome
Describe recovery, reconciliation, and prevention

Continue core path Explore all production scenarios

Stage 7

0/4 core exercises

System design

Turn requirements into defensible architectures with reliability, replay, quality, and cost trade-offs.

Design an e-commerce orders data platform

Batch architecture, auditability and revenue reconciliation

Beginner · 35 min

Design Postgres CDC to a warehouse

Change capture, ordering and replay

Intermediate · 40 min

Design a data-quality monitoring system

Unlock Premium to open this exercise.

Design a backfill-safe pipeline

Unlock Premium to open this exercise.

Ready to move on when you can

Clarify scale, latency, consumers, and data contracts
Defend architecture choices and their trade-offs
Include observability, replay, security, and failure recovery

Continue core path Explore all system-design cases

Ongoing review loop

Reattempt weak areas instead of collecting completion badges.

After every few exercises, use the dashboard to revisit failed attempts, explanations with gaps, and concepts that did not hold up on the first try. Review remains ongoing and is not presented as a final readiness claim.

Review practice evidence