Data Engineering

Data platform decisions that age well

The choices that hurt three years in are rarely tool choices. They are ownership, metric definitions and contract boundaries between systems.

21 April 20269 min read

Platform regret usually is not about the warehouse product. It shows up as three unreconciled definitions of revenue, a pipeline nobody owns, and a dashboard that stakeholders no longer trust enough to act on.

The durable decisions are organisational as much as technical: who owns a dataset, what a metric means, and what guarantees a producing system makes to its consumers.

Agree metric definitions with the business first

Write the definition of each headline metric — with edge cases, exclusions and the system of record — and have the commercial owner sign it. This single artefact ends most recurring dashboard arguments.

Treat interfaces as contracts

Where a system publishes data, define the schema, freshness expectation and breaking-change process. Contracts make pipeline failures a producer conversation rather than an analytics firefight.

Model for questions, not for tables

Start from the decisions the platform must support and work back to the modelling. Warehouses that mirror source schemas tend to push complexity onto every consumer forever.

Make quality visible

  • Tests on the assumptions that matter: uniqueness, referential integrity, freshness, range
  • Alerting routed to the owning team, not a shared inbox
  • A published status view so consumers can see when data is late or partial

Every engagement starts with a
45-minute strategy session.

Bring the problem, the constraints and the deadline. You leave with an architecture opinion, a delivery shape and a realistic budget range — no pitch deck.

Book strategy session