System DesignAdvancedInterview ExplanationFree

The Dual-Write Migration

You are the data engineer on call for this production path. A legacy and new pipeline run in parallel, but their outputs diverge and cutover confidence falls. What would you investigate, how would you fix it safely, and how would you prove the issue is resolved?

Scenario context

The incident centers on parallel-run migration validation. The current implementation or operating process does not make that contract explicit, so the team needs a diagnosis supported by evidence rather than a tool or configuration guess.

Business requirement

Identify the most likely failure mechanism, propose a reversible production-safe fix, and explain validation, trade-offs, monitoring, and recovery.

Schema

Architecture & Leadership evidence is shown below. Treat it as a production review artifact rather than a toy exercise.

Broken logic / code

legacy_pipeline -> legacy_output
new_pipeline    -> new_output
row_count_delta=0.3%
revenue_delta=2.8%
cutover_threshold=UNDEFINED
rollback_owner=UNASSIGNED

Logs / error

[Production review] Scenario 229: The Dual-Write Migration
Observed symptom: A legacy and new pipeline run in parallel, but their outputs diverge and cutover confidence falls. What would you investigate, how would you fix it safely, and how would you prove the issue is resolved?
Core contract at risk: parallel-run migration validation.
Evidence to collect: Track adoption, delivery lead time, incident recurrence, migration mismatch rate, cutover readiness, capacity headroom, and action-item closure. Add architecture game days and operational-readiness reviews. Run consumer value and actual usage, not only platform output.

Actual output

A legacy and new pipeline run in parallel, but their outputs diverge and cutover confidence falls. What would you investigate, how would you fix it safely, and how would you prove the issue is resolved?

Expected output / expected logic

A strong response should define the contract, rank likely causes, propose a safe fix, and prove correctness with monitoring and reconciliation.

Your attempt

Write your answer

Think before revealing the answer. A partial but honest attempt is better practice than reading the model solution first.

Saved

Interview-style explanation

Now explain your solution as if you are in an interview: symptom, root cause, fix, edge cases, trade-offs, monitoring, and prevention.