← All posts

When a clean API becomes too clean

Building the Plan.Make.Trace frontend has started to push back on Dawnshard. This post covers the first two core changes that came from it: join rows that we deliberately removed from API responses and then had to put back, and a tighter contract for what it means to clear a field.

What we built

Two corrections to Dawnshard, both found by the frontend using the API for real rather than by anything we could see from the backend alone.

The first is that a record reached through a many-to-many include now comes back carrying its join row. The second is a single rule for validation: undefined means omit, null means clear, and a required field can never be cleared.

The problem it solves

When a many-to-many relationship is included in a response, each target record used to arrive without the join row that linked it. I stripped it out on purpose. It looked like relationship plumbing the client had no reason to see, and a response without it was tidier.

That was premature optimisation, and I was the one optimising. The frontend data layer breaks responses apart and writes them into a normalised local store, and for a many-to-many relationship the join row is the link. It holds the key of the record and the key of its target. Without it the store could hold both records and have no way to say they were related.

So I spent time removing information from a response, and then spent more time putting the shit back.

The challenges

The join row has no access config of its own, and we did not want to invent one. The answer was to treat it as part of the association. It rides on a target record that has already passed its own scope check, so it is authorised by that check. If the target is out of scope, the target goes and the join row goes with it. A root record never keeps a join row, because the root is where create and update payloads are filtered.

The second constraint is that the join row comes back whole. That is fine for a pure joiner, which is an id and two foreign keys. It is not fine for a join table that carries its own data, because declaring that as a `via` would let its extra fields through unfiltered. A join table with its own data has to be a full service, reached through a normal one-to-many include and field-shaped like anything else. That rule is now written next to the code that depends on it.

The validation fix had a different shape. Clearing used to be an opt-in: each field type had its own `nullable` handling, and a fallback type had its own test to reject null. Every one of those is a place to get the same rule slightly wrong. This was a correctness problem, not an optimisation one.

What makes ours different

There is a good case for stripping join rows. Plenty of APIs hide their join tables, and for a client that only displays nested records it is the right call. Ours has a client that normalises, and that client needs the link.

The join keys are worked out once at boot from every model's associations, not per request, so the response filter does no extra lookups. Boot also fails if a `via` names a model that was never registered. That is why `featureFlagAccount`, `featureFlagProfile` and `roleBundleRole`, which had never been registered, now exist as small model-only modules. A typo in a `via` fails at startup, not on someone's screen.

On the validation side the rule now lives in one shared place instead of in every type branch. Optional fields are clearable by default. `required` rejects null, demands a value on create and still allows omission on a partial update. `requiredOnCreate` demands a value on create and allows clearing afterwards. Around thirty core models were updated so that fields which must always hold a value say `required`.

A contract test runs that rule across fourteen field types, and the number types are tested to reject empty strings, NaN and junk, with inclusive bounds.

What this unlocks

The frontend store can link many-to-many records straight from an include response, with no second request to find out what belongs to what. Realtime and scopes work from the same normalised rows, so the link only has to be right once.

Model authors get a clearer deal as well. Send null to unlink a record or clear a date, leave the key out to leave it alone, and a required field refuses to be emptied. Neither side has to guess which of those a payload meant.

The wider point is that Dawnshard is now being shaped by a real client. It is the first time the core has changed because a screen needed something, and I would rather find these gaps now than after the beta.

What's next

The formula workspace. It is close to complete, including the new scent properties, and it is the first major screen built against the visual language from the last post.

This post is late because my beautiful cat died. RIP Miss Cracker.

The amazing Miss Cracker who sadly passed this week. At 23 years old, she had a good run.
The amazing Miss Cracker.

Liked this post? Tap the heart to let us know.

Get early access

Be the first to hear about early access, roadmap updates, and exclusive early adopter pricing.

Join the early access list