If you serialise objects to JSON anywhere — a database row, a cache entry, a queue message — this bug is probably in your code right now, and it is not throwing.
const isPlain = (v) =>
v !== null && typeof v === 'object' && !Array.isArray(v);
That predicate looks unremarkable. It is how most serialisers decide "is this an
object whose keys I should walk?" And it is true for ArrayBuffer, every typed
array, DataView, Blob, File, Map, Set, RegExp, Error and URL —
none of which keep their data in enumerable own properties.
So the walker walks them, finds nothing, and stores nothing. JSON.stringify
does not throw on any of it.
I found this in my own library, in a file I had written specifically to stop this class of bug.
The setup, so the failure makes sense
granthdb is SQLite compiled to WebAssembly,
running in the browser behind a Dexie-compatible API. Documents are stored as
JSON in a column, with declared indexes as generated columns over
json_extract — that is what lets SQLite's planner do real index work on what
is, underneath, a document store.
Storing documents as JSON has one obvious problem, and I knew about it. IndexedDB
— the thing granthdb replaces — stores values with the structured clone
algorithm, which preserves Date, NaN, Infinity, BigInt and undefined.
JSON preserves none of them: a Date becomes a string, NaN and Infinity
become null, BigInt throws, undefined makes the key vanish.
For a library claiming Dexie compatibility, that is data corruption rather than a
limitation. So there is a codec. It has a comment at the top saying exactly that.
It handles Date, NaN, Infinity, BigInt, undefined and null, encoding
each as a sentinel-prefixed string so the stored shape stays a scalar and the
indexes keep working.
It handled six types. It destroyed every other type in the structured clone algorithm, silently, for months.
What I actually got back
I only looked because someone asked whether granthdb was any good for storing PDFs. Before answering I wrote eight lines to check what happens today:
await db.files.add({ name, body: value });
const back = (await db.files.get(id)).body;
console.log(back?.constructor?.name, JSON.stringify(back));
| I stored | I got back |
|---|---|
Uint8Array([37,80,68,70]) | {"0":37,"1":80,"2":68,"3":70} — an Object |
ArrayBuffer | {} |
Blob, File | {} |
Map, Set | {} |
RegExp | {} |
Two different failures, one cause.
A Uint8Array does have enumerable own properties — its indices. So the
walker walked it and produced an object with numeric keys: the right bytes, the
wrong type, and roughly nine times the size once each byte became "0":37,.
Code doing bytes instanceof Uint8Array got false. Code doing bytes[0] got
37 and carried on, which is worse.
An ArrayBuffer, a Blob and a Map have no enumerable own properties.
They walked to {}. The data was gone at the moment of the write, and nothing
anywhere reported it.
The leaf encoder was never the problem. The predicate above it claimed the value first, so the walker consumed it before any branch that knew what it was could run.
Why nothing threw
This is the part worth internalising, because it generalises past my library.
JSON.stringify has exactly one loud failure: a circular reference. Everything
else it handles by quietly producing something. A function becomes undefined.
A Symbol key disappears. An object with no enumerable properties becomes {},
which is a completely valid JSON document.
So the write succeeded. The row was valid. The schema was satisfied. Every test passed, because every test asserted on data shapes that happened to be plain objects, numbers and strings.
The failure mode of a serialiser is not an exception. It is a smaller, valid document. That is why you cannot rely on your error handling to find these — you have to enumerate the types deliberately and assert the round trip.
The reference list you should be checking against
If your thing claims to be a store — and especially if it claims IndexedDB or Dexie compatibility — the contract is the structured clone algorithm. IndexedDB stores values with it, so whatever structured clone preserves is what a caller has every reason to expect back.
Enumerate against that list, not against imagination. It also tells you what
should fail: URL is not cloneable, so IndexedDB throws DataCloneError on
one. Storing {} instead is still a divergence, just a less severe one than
dropping data the platform would have kept.
The fix has three parts, and missing one leaves it broken
I got this wrong on my first pass, in a way that took a while to see.
I added a branch to the leaf encoder that threw a clear, helpful error when it
met a Blob. Good message, named the fix, told you to call .arrayBuffer().
It never fired. Not once.
Because isPlain still claimed the Blob, so the walker still consumed it
first, and the branch I had just written sat below code that never reached it. I
had written an error message into dead code and — briefly — believed the bug was
fixed.
Each part is individually obvious. The trap is that doing two of the three produces code that looks finished, and in the case of part 2 produces an error message that can never be printed.
So, in order:
- Encode the constructor name, not just the payload. A
Float64Arrayand aUint8Arrayover identical bytes are different values. Decoding to the wrong one is a silent numeric change, not an error. - Exclude the type from the "plain object" predicate. Otherwise part 1 is unreachable.
- Recurse into containers. A
Mapwhose entries are not themselves encoded fixes the container and leaves the identical bug one level down. AMapofDates must survive as aMapofDates.
Two traps inside the encoding itself
A view onto a slice must store its own bytes.
const view = new Uint8Array(bigBuffer, 8, 4); // four bytes
Encode view.buffer and you have persisted the entire backing buffer and will
read back the wrong length. subarray(0, 4) of a 10 MB buffer silently writes
10 MB. Use byteOffset and byteLength.
Base64 that only works on fixtures. The idiomatic one-liner is:
btoa(String.fromCharCode(...bytes))
That spreads every byte as a separate function argument. It blows the call stack somewhere around 100 kB — which means it passes every test written against a small fixture and throws on the first real file a user picks. Chunk it:
let s = '';
for (let i = 0; i < bytes.length; i += 0x8000) {
s += String.fromCharCode(...bytes.subarray(i, i + 0x8000));
}
return btoa(s);
This is the same species of bug as the main one: correct against the data you thought to test, wrong against the data you will actually receive.
Some types cannot be fixed, only refused
Blob and File only yield their bytes through await blob.arrayBuffer(). My
codec is synchronous, called from inside the write path. It cannot await.
There were two real options: convert them earlier, where await is available, or
throw. I threw — with a message that names the fix:
granth: cannot store a Blob or File directly — reading its bytes is
asynchronous and this runs inside the write path. Pass the bytes instead:
const bytes = new Uint8Array(await file.arrayBuffer());
Throwing is not a cop-out here. It replaces silent total loss with a failure at the exact line that caused it. The worst outcome available was the one already shipping.
One detail: the check is duck-typed rather than instanceof Blob.
typeof v.arrayBuffer === 'function' && typeof v.size === 'number'
A value structured-cloned across a worker boundary is not always an instance of
the receiving realm's constructor. instanceof is a realm-local question, and
this code runs in a dedicated Worker, in Node under an inline runtime, and in a
SharedWorker.
The half-fix I nearly shipped
I fixed the binary types, wrote the tests, and was about to publish.
Then I checked whether Map, Set and RegExp had the same problem. They did —
same predicate, same silence, and all three are in the structured clone algorithm
too.
Shipping the binary fix alone would have been worse than shipping nothing.
"Binary works now" is a claim that sends the next person to debug their vanishing
Map somewhere else entirely — into their own code, into the query layer,
anywhere but the codec they were just told had been fixed.
Error I deliberately left out, and said so in the docs. It is
structured-cloneable, but a round-tripped Error loses its stack and gains a
different prototype. Restoring one is a half-truth, and storing an exception in a
queryable row is usually a modelling mistake anyway.
The guard, and making sure it can fail
The test is 42 assertions across every typed array, DataView, ArrayBuffer,
nested and recursive cases, a 1 MB buffer for the base64 chunking, empty
collections, and the two refusals.
Then I reverted the isPlain change and ran it again. It failed on the first
assertion:
AssertionError: ArrayBuffer: constructor
+ actual - expected
+ 'Object'
- 'ArrayBuffer'
That step is not ceremony. A test you have never watched fail is a test you are hoping about. I had already been burned on that exact point in the same codebase: a compatibility audit with a hand-maintained list of "things we deliberately don't implement" that could only ever check one direction, so an entry claiming a method was missing — while it had been implemented all along — was never evaluated and never contradicted. Green every run, for months.
Where a PDF actually belongs
The question that started this deserves its answer.
Binary in a row is stored base64 inside the document's JSON. That is +33%, and the whole document is re-parsed on every read of that row. I measured it: 1 MB of bytes occupies 1.33 MB in the row.
That is the right trade for an avatar, a signature, a thumbnail or an icon. It is the wrong trade for a PDF, a video or a model file. Those belong beside the database, not inside it: bytes streamed to OPFS, metadata in the queryable store.
The split is the same one the database itself makes: one place that is good at queries, one that is good at bytes.
The takeaway
typeof x === 'object' answers a question you are not asking. You want to know
"is this a plain object", and the language has no operator for that — so every
serialiser encodes its own guess, and every guess is a list of exceptions that is
complete on the day it is written.
Three things I would do differently, and now do:
- Enumerate against a spec, not against the types you happen to use. For browser storage that spec is structured clone. For a queue or a cache it might be your wire format's own type list. Either way, write the list down.
- Assert the round trip per type, including the constructor.
toEqualon a plain-object comparison will happily pass on{"0":37}. - Revert the fix and watch the test fail. A serialiser bug produces valid output, so a test that has never gone red is telling you nothing.
The full write-up of what granthdb does with each type — and where to put the bytes when a row is the wrong place — is in Files and binary data.
npm install granthdb @sqlite.org/sqlite-wasm
- Package — granthdb on npm
- Documentation — granthlabs.github.io
- Source — granthlabs/granth
Why it exists at all is in this post, and the project write-up is at /case-studies/granthdb-browser-sqlite.
If you're carrying a similar problem in a system you'd rather not describe in public, that's the kind of thing I audit — the 185-table Postgres audit is the same method pointed at a database instead of a codec.