Skip to content

field_note · aug 24

The bug that stored your file as {}: typeof x === 'object' is not a plain object

I wrote a codec specifically to stop JSON silently mangling values, and it destroyed every type I forgot to list. Blob, ArrayBuffer, Map, Set and RegExp all stored as an empty object, with nothing thrown. Why serialisers fail in the direction of silence, and the three-part fix where missing one part leaves it broken.

If you serialise objects to JSON anywhere — a database row, a cache entry, a queue message — this bug is probably in your code right now, and it is not throwing.

terminal
const isPlain = (v) =>
  v !== null && typeof v === 'object' && !Array.isArray(v);

That predicate looks unremarkable. It is how most serialisers decide "is this an object whose keys I should walk?" And it is true for ArrayBuffer, every typed array, DataView, Blob, File, Map, Set, RegExp, Error and URL — none of which keep their data in enumerable own properties.

So the walker walks them, finds nothing, and stores nothing. JSON.stringify does not throw on any of it.

I found this in my own library, in a file I had written specifically to stop this class of bug.

The setup, so the failure makes sense

granthdb is SQLite compiled to WebAssembly, running in the browser behind a Dexie-compatible API. Documents are stored as JSON in a column, with declared indexes as generated columns over json_extract — that is what lets SQLite's planner do real index work on what is, underneath, a document store.

Storing documents as JSON has one obvious problem, and I knew about it. IndexedDB — the thing granthdb replaces — stores values with the structured clone algorithm, which preserves Date, NaN, Infinity, BigInt and undefined. JSON preserves none of them: a Date becomes a string, NaN and Infinity become null, BigInt throws, undefined makes the key vanish.

For a library claiming Dexie compatibility, that is data corruption rather than a limitation. So there is a codec. It has a comment at the top saying exactly that. It handles Date, NaN, Infinity, BigInt, undefined and null, encoding each as a sentinel-prefixed string so the stored shape stays a scalar and the indexes keep working.

It handled six types. It destroyed every other type in the structured clone algorithm, silently, for months.

What I actually got back

I only looked because someone asked whether granthdb was any good for storing PDFs. Before answering I wrote eight lines to check what happens today:

terminal
await db.files.add({ name, body: value });
const back = (await db.files.get(id)).body;
console.log(back?.constructor?.name, JSON.stringify(back));
I storedI got back
Uint8Array([37,80,68,70]){"0":37,"1":80,"2":68,"3":70} — an Object
ArrayBuffer{}
Blob, File{}
Map, Set{}
RegExp{}

Two different failures, one cause.

A Uint8Array does have enumerable own properties — its indices. So the walker walked it and produced an object with numeric keys: the right bytes, the wrong type, and roughly nine times the size once each byte became "0":37,. Code doing bytes instanceof Uint8Array got false. Code doing bytes[0] got 37 and carried on, which is worse.

An ArrayBuffer, a Blob and a Map have no enumerable own properties. They walked to {}. The data was gone at the moment of the write, and nothing anywhere reported it.

Where a value falls through the encoderencode(value)isPlain(value)?typeof v === 'object'truewalk its keysObject.entries(v){"0":37,"1":80,…}{}falseencodeValue(v)the leaf encoderDate · NaN · Infinity · BigInt · undefined · null — all handledBlob, ArrayBuffer, Map, Set and RegExp never reach the leaf encoder.The branch that would have handled them is above the branch that swallows them.Adding a case to encodeValue() alone changes nothing at all.

The leaf encoder was never the problem. The predicate above it claimed the value first, so the walker consumed it before any branch that knew what it was could run.

Why nothing threw

This is the part worth internalising, because it generalises past my library.

JSON.stringify has exactly one loud failure: a circular reference. Everything else it handles by quietly producing something. A function becomes undefined. A Symbol key disappears. An object with no enumerable properties becomes {}, which is a completely valid JSON document.

So the write succeeded. The row was valid. The schema was satisfied. Every test passed, because every test asserted on data shapes that happened to be plain objects, numbers and strings.

The failure mode of a serialiser is not an exception. It is a smaller, valid document. That is why you cannot rely on your error handling to find these — you have to enumerate the types deliberately and assert the round trip.

The reference list you should be checking against

If your thing claims to be a store — and especially if it claims IndexedDB or Dexie compatibility — the contract is the structured clone algorithm. IndexedDB stores values with it, so whatever structured clone preserves is what a caller has every reason to expect back.

Enumerate against that list, not against imagination. It also tells you what should fail: URL is not cloneable, so IndexedDB throws DataCloneError on one. Storing {} instead is still a divergence, just a less severe one than dropping data the platform would have kept.

The fix has three parts, and missing one leaves it broken

I got this wrong on my first pass, in a way that took a while to see.

I added a branch to the leaf encoder that threw a clear, helpful error when it met a Blob. Good message, named the fix, told you to call .arrayBuffer().

It never fired. Not once.

Because isPlain still claimed the Blob, so the walker still consumed it first, and the branch I had just written sat below code that never reached it. I had written an error message into dead code and — briefly — believed the bug was fixed.

The three parts of the fix, and what breaks if you miss one1. Encode the type\0Y<Ctor>,<base64>miss it → Float64Array decodes as Uint8Array. Silent numeric change.2. Exclude from isPlain!isBinary && !isBlobLikemiss it → part 1 is dead code. The walker gets there first.3. Recurse into itencode([...map])miss it → the Map survives, the Date inside it does not.

Each part is individually obvious. The trap is that doing two of the three produces code that looks finished, and in the case of part 2 produces an error message that can never be printed.

So, in order:

  1. Encode the constructor name, not just the payload. A Float64Array and a Uint8Array over identical bytes are different values. Decoding to the wrong one is a silent numeric change, not an error.
  2. Exclude the type from the "plain object" predicate. Otherwise part 1 is unreachable.
  3. Recurse into containers. A Map whose entries are not themselves encoded fixes the container and leaves the identical bug one level down. A Map of Dates must survive as a Map of Dates.

Two traps inside the encoding itself

A view onto a slice must store its own bytes.

terminal
const view = new Uint8Array(bigBuffer, 8, 4);   // four bytes

Encode view.buffer and you have persisted the entire backing buffer and will read back the wrong length. subarray(0, 4) of a 10 MB buffer silently writes 10 MB. Use byteOffset and byteLength.

Base64 that only works on fixtures. The idiomatic one-liner is:

terminal
btoa(String.fromCharCode(...bytes))

That spreads every byte as a separate function argument. It blows the call stack somewhere around 100 kB — which means it passes every test written against a small fixture and throws on the first real file a user picks. Chunk it:

terminal
let s = '';
for (let i = 0; i < bytes.length; i += 0x8000) {
  s += String.fromCharCode(...bytes.subarray(i, i + 0x8000));
}
return btoa(s);

This is the same species of bug as the main one: correct against the data you thought to test, wrong against the data you will actually receive.

Some types cannot be fixed, only refused

Blob and File only yield their bytes through await blob.arrayBuffer(). My codec is synchronous, called from inside the write path. It cannot await.

There were two real options: convert them earlier, where await is available, or throw. I threw — with a message that names the fix:

terminal
granth: cannot store a Blob or File directly — reading its bytes is
asynchronous and this runs inside the write path. Pass the bytes instead:
  const bytes = new Uint8Array(await file.arrayBuffer());

Throwing is not a cop-out here. It replaces silent total loss with a failure at the exact line that caused it. The worst outcome available was the one already shipping.

One detail: the check is duck-typed rather than instanceof Blob.

terminal
typeof v.arrayBuffer === 'function' && typeof v.size === 'number'

A value structured-cloned across a worker boundary is not always an instance of the receiving realm's constructor. instanceof is a realm-local question, and this code runs in a dedicated Worker, in Node under an inline runtime, and in a SharedWorker.

The half-fix I nearly shipped

I fixed the binary types, wrote the tests, and was about to publish.

Then I checked whether Map, Set and RegExp had the same problem. They did — same predicate, same silence, and all three are in the structured clone algorithm too.

Shipping the binary fix alone would have been worse than shipping nothing. "Binary works now" is a claim that sends the next person to debug their vanishing Map somewhere else entirely — into their own code, into the query layer, anywhere but the codec they were just told had been fixed.

Error I deliberately left out, and said so in the docs. It is structured-cloneable, but a round-tripped Error loses its stack and gains a different prototype. Restoring one is a half-truth, and storing an exception in a queryable row is usually a modelling mistake anyway.

The guard, and making sure it can fail

The test is 42 assertions across every typed array, DataView, ArrayBuffer, nested and recursive cases, a 1 MB buffer for the base64 chunking, empty collections, and the two refusals.

Then I reverted the isPlain change and ran it again. It failed on the first assertion:

terminal
AssertionError: ArrayBuffer: constructor
+ actual   - expected
+ 'Object'
- 'ArrayBuffer'

That step is not ceremony. A test you have never watched fail is a test you are hoping about. I had already been burned on that exact point in the same codebase: a compatibility audit with a hand-maintained list of "things we deliberately don't implement" that could only ever check one direction, so an entry claiming a method was missing — while it had been implemented all along — was never evaluated and never contradicted. Green every run, for months.

Where a PDF actually belongs

The question that started this deserves its answer.

Binary in a row is stored base64 inside the document's JSON. That is +33%, and the whole document is re-parsed on every read of that row. I measured it: 1 MB of bytes occupies 1.33 MB in the row.

That is the right trade for an avatar, a signature, a thumbnail or an icon. It is the wrong trade for a PDF, a video or a model file. Those belong beside the database, not inside it: bytes streamed to OPFS, metadata in the queryable store.

Bytes inside a row versus bytes beside the rowinside the row1 MB of bytes → 1.33 MB of base64 in _docevery get() on this row re-parses all of itbeside the rowrow: 200 bytesname · type · size · keyOPFS file: 1 MBstreamed, never encodedqueries stay on the row.orderBy('updated').sum('size')Same total bytes on disk. The difference is what a single read has to touch.

The split is the same one the database itself makes: one place that is good at queries, one that is good at bytes.

The takeaway

typeof x === 'object' answers a question you are not asking. You want to know "is this a plain object", and the language has no operator for that — so every serialiser encodes its own guess, and every guess is a list of exceptions that is complete on the day it is written.

Three things I would do differently, and now do:

  • Enumerate against a spec, not against the types you happen to use. For browser storage that spec is structured clone. For a queue or a cache it might be your wire format's own type list. Either way, write the list down.
  • Assert the round trip per type, including the constructor. toEqual on a plain-object comparison will happily pass on {"0":37}.
  • Revert the fix and watch the test fail. A serialiser bug produces valid output, so a test that has never gone red is telling you nothing.

The full write-up of what granthdb does with each type — and where to put the bytes when a row is the wrong place — is in Files and binary data.

terminal
npm install granthdb @sqlite.org/sqlite-wasm

Why it exists at all is in this post, and the project write-up is at /case-studies/granthdb-browser-sqlite.

If you're carrying a similar problem in a system you'd rather not describe in public, that's the kind of thing I audit — the 185-table Postgres audit is the same method pointed at a database instead of a codec.

Working through something like this? I help teams ship AI and cloud systems that hold up, and cost what they should.