Data engineering insights for growing businesses
What we have learned building data platforms for SMEs and startups — written for the people who pay for them, not only the people who run them.
21 posts
24 Jul 2026
Extracting data from unstructured documents: design before tooling
PDFs hold the majority of an organisation's unstructured data, and the instinct is to point an LLM at them. That instinct has real limits. The right approach starts with the use case, the structure, and the content - not the tool.
Data acquisition · Operational efficiency · Finance and operations
10 Apr 2026
The AI governance conversation is older than you think
Boards faced with AI disruption need to thread a needle of "Move fast, safely". The domain may be new and the surface area wider, but the principles are not. Human processes, third-party dependencies, and data governance form precedents for Directors.
Leadership · Risk reduction
9 Apr 2026
Data decrepitude and guarding against drift
Your data mess didn't happen overnight. It was slow drift: one spreadsheet, one stale dashboard, one unanswered ticket at a time. We break down why data decrepitude happens and what you can do about it.
Data strategy · Operational efficiency · Leadership
2 Apr 2026
Don't train your AI Agents to repeat the past
Meta's analytics agent is a serious piece of engineering. But most orgs aren't Meta. They're trying to build analytical capability from a standing start and here the highest-leverage isn't the agent, it's the semantics that let anyone ask questions.
Operational efficiency · Leadership · Data modelling
7 Mar 2025
The data edge in Private Equity: From acquisition to value creation
PE firms are increasingly relying on data and AI for competitive advantage. From deal sourcing to post-acquisition, investments in data maturity enhance productivity and drive value creation for PE firms and portfolio companies
Data strategy · Leadership · Revenue growth
9 Dec 2024
Resolving the Build vs Buy Dilemma: Approaching Data Pipeline Decisions with Tact
The build vs buy decision is a strategic one. Discover a nuanced framework that helps technology leaders make smarter infrastructure choices that drive real competitive advantage.
Data infrastructure · Engineering · Startups
9 Sept 2024
Optimizing Data Warehouse Costs with Query Parsing
This article covers how query parsing can optimize data warehouse costs by identifying inefficient SQL patterns, using Abstract Syntax Trees (ASTs), and tailoring query refactoring for better performance.
Cost optimisation · Data infrastructure · Engineering
4 Sept 2024
MDS in a box
For SME and startups, anything that allows you to spend more time creating value and less time on 'busy work' is helpful. We've put together a sample "Modern Data Stack" in a box to prototype from data to pipeline to analysis rapidly and at low cost
Data infrastructure · Engineering · Operational efficiency
30 Aug 2024
Automate the Analyst: Part 2 - All about context
Your analyst is still a key part of the insight engine that helps you power decisions in your business, but GenAI has all the components to complement that value.
Leadership · Data modelling · Operational efficiency
19 Jul 2024
Automate the Analyst: Part 1 - Building a Robust Data Foundation
Part 1 of the series talks about the foundations that maximize your returns on AI investments through data capture, quality and lineage information
Leadership · Data strategy · Data infrastructure
21 Jun 2024
Tracking data platform usage is critical
Your team has invested significant time and money into building data infrastructure and creating an array of data assets - raw data tables, transformed data sets, dashboards, machine learning models, and more. But has this activity truly generated value?
Data modelling · Data infrastructure · Cost optimisation
15 May 2024
Mapping Business Value to your Cost Optimization Practices
Cost Management Practices are often haphazard. They fail to identify business value of data workloads, and therefore fail to decide between replacing or fine-tuning them. This article lays out our approach to thoughful cost management.
Cost optimisation · Leadership · Data modelling
6 May 2024
Transformation tooling: SQL Mesh vs dbt core
dbt's approach to transformations democritized data warehousing and made itself central to modern data platform designs. In recent times, SQLMesh has emerged as an alternative and this article compares both tools
Data modelling · Engineering · data build tool
25 Apr 2024
Assessing your data maturity
Companies naturally go through stages of data maturity as they grow. This guide provides a broad set of opinions that we hold on what the data maturity curve looks like for typical organisations, along with guidance for senior executives on transitions
Org and process · Leadership
12 Apr 2024
Benchmarking Ingestion across Analytical Database Destinations
Latency and performance of upload is a non-trivial consideration when building your analytical database. This article shows you results from some of our testing.
Engineering · Data acquisition · Data infrastructure
10 Apr 2024
data load tool (dlt): An introduction
Every now and again there is a tool that takes a different approach to loading data in scalable, manageable ETL pipelines, and we’ve liked what we've seen so far from data load tool, which is a fresh, open core approach to solving this problem
Engineering · Data infrastructure
14 Mar 2024
Is my data stack appropriate?
It's really hard to know whether your current data stack is appropriate for the scale of operations. While there is no template to follow, this article outlines what we've seen is typical amongst companies at various stages of their development
Data infrastructure · Engineering · Operational efficiency
12 Mar 2024
Looker: Understanding the building blocks
Following our introductory post on data modelling to help answer the all-important 'why' questions about your business, we explore the building blocks of Looker and help guide how to arrange these into a structure that will grow with you
Looker · Data modelling · Operational efficiency
12 Mar 2024
Data modelling: Creating structures that grow with your business
Imagine you are the owner of a successful ecommerce company. At first glance, describing your operations seems straightforward. But what really drives performance under the hood? Answering questions like these quickly moves into understanding causality
Data modelling · Operational efficiency · Leadership
1 Mar 2024
Analytics Engineering: Building Data like Software
This article introduces the discipline of analytics engineering, exploring its benefits in maintaining content to drive decision-making. We also introduce you to one of the most popular tools that help with this: DBT (data build tool).
Data modelling · Leadership · data build tool
1 Mar 2024
How to think about Data Platform Cost
Data Platform costs can often balloon up if left unchecked. In the period of (relatively higher) austerity, it's helpful to develop a framework and strategy to reduce your costs.
Cost optimisation · Finance and operations · Data infrastructure