← All posts

Building the frontend data layer

Every screen in Plan.Make.Trace reads from a local store, not directly from the API. Server responses are broken apart and written into that store by model, so a recipe and its ingredients do not travel together once they arrive. This post covers how that data layer works and why it is built the way it is.

What we built

The frontend data layer has three distinct parts: a DataService, a DataStore, and the machinery that connects them to the rest of the application.

The DataService is the boundary over HTTP. Domain code calls a consistent set of operations against named resources and the transport lives entirely behind it. No domain imports Axios. The DataStore is the boundary over local persistence. Domain code reads streams of records and the IndexedDB engine lives entirely behind it. No domain imports Dexie.

Between those two boundaries sits the shredder. When a response arrives from the API, it is a composed tree: a company with its partnerships nested inside it, or a batch with its ingredient allocations. The shredder walks that tree, pulls each nested record out, and files it into the appropriate model bucket. A company goes into the company table, its partnerships go into the partnership table, each flat and keyed by their own id. Everyone loves Shredder from TMNT, so the name stuck.

Above all of that is the domain store factory. Every domain service is built from it: three values differ between domains, the API path, the model name, and which associations to include, and the rest is shared. Fetch, track, shred, observe.

The problem it solves

TanStack Query is the obvious reference point for frontend server-state management, and it is a good tool. But its cache is organised around queries and their keys. Plan.Make.Trace needed something different: records from composed API responses needed to become shared objects in a normalised local store, while the system separately tracked whether particular result sets were complete and current.

Plan.Make.Trace works differently. Responses are shredded into a normalised local store, and individual records participate in many different queries. A partnership record fetched as part of a company response and the same partnership fetched directly are the same row in the same table. There is no second copy to invalidate.

The challenge with normalisation is knowing what the store holds completely. Records being present is not the same as a query being satisfied. You can have ten partnership records in the store and still not know whether that is all of them for a given company, or just the ones that arrived as includes on earlier reads.

Scopes are the answer to that. A scope records that a particular filter has been fully fetched. When the store is asked for the partnerships belonging to a company, it checks whether that scope is satisfied before deciding whether to go to the network. Satisfied means the stream from the local store is the answer. Unsatisfied means a fetch runs, the results are shredded in, and the scope is confirmed once the last page lands.

The challenges

The hardest part was getting the scope and realtime systems to work together without duplicating the logic between them.

Realtime delivers dirty batches from the backend: a set of model names and record ids that changed since the last flush. The straightforward response would be a separate reconciliation path that handled realtime differently from the normal fetch flow. That would have produced two implementations of the same logic, and two implementations of anything is two chances to get one of them wrong.

Instead, a dirty realtime batch translates directly into scope invalidation. A deleted record gets tombstoned and removed from the store. An updated record marks its scope dirty. A dirty scope is the same signal the domain store's fetch loop already watches. There is no realtime-specific downstream of that point.

The tombstone mechanism closes a specific race: a delete fires and a response from a concurrent in-flight read lands after it. Without the tombstone, that response would write the deleted record back. The tombstone guards the id for the duration of the HTTP timeout, blocking any write for it regardless of when that response arrives.

What makes this different

The local store's schema is not hardcoded in the frontend. It is built at login from the model definitions the backend sends down. The same definitions that drive backend validation, access control, and routing are what the frontend uses to declare its IndexedDB tables and their indexes.

A new domain added to the backend arrives in the frontend store automatically. The right tables, the right indexes, no frontend schema to update. The store version is a hash of those definitions, so a model shape change produces a new database rather than a migration.

The normalised shape means associations fetched as part of one query are available to any other query that needs them. Partnerships fetched with a company do not just populate that company screen. Those partnership records are in the partnership table, available to anything that reads partnerships by company, without a second request.

What this unlocks

Plan.Make.Trace is intended to be used where the work is actually happening. Someone may be receiving inventory while somebody else is preparing a batch, consuming materials, updating production, or working from another device elsewhere in the workshop. Those actions need to become visible where they matter without everyone refreshing screens to find out what changed. Especially when you have fast batch and all you want to see is if you remembered to add the god dam sodium lactate.

That was one of the original reasons for building the frontend data layer this way. I was tired of SaaS applications that feel disconnected from the work happening around them: make a change, switch screens, refresh, wait, and hope you are looking at the latest state. For a manufacturing application built around coordination and traceability, that was not good enough.

The normalised store, scopes, and realtime invalidation work together so the application can respond consistently to changes regardless of where they originated. A stock change can appear where that stock matters. A batch update can propagate to the screens depending on it. The user does not need to understand where the data came from or manually ask the application whether anything has changed.

The goal is simple: when Plan.Make.Trace is being used on the factory floor, the software should keep up with the work.

What's next

The frontend data layer is the foundation the product screens sit on. The next post covers the production batch workflow: how Plan.Make.Trace takes a recipe through manufacturing, connects the stock it consumes, and produces the finished goods that go out.

Liked this post? Tap the heart to let us know.

Get early access

Be the first to hear about early access, roadmap updates, and exclusive early adopter pricing.

Join the early access list