MetalBear closes its reference-parity gap on server, admin, and repo writes

MetalBear

Most of this stretch was a parity audit: walking the reference PDS's TypeScript endpoint by endpoint and fixing whatever MetalBear did differently. Alongside it, server.c and repo_store.c — both several thousand lines — got split into domain-scoped files, and the repo/blob stores gained real mutex protection for a server that now runs a real thread pool instead of serializing everything by accident.

concurrency: real threads, real races

metalbear_repo_store and metalbear_blob_store had no locking at all until feat(repo): thread-safe repo store and blob store added a pthread_mutex_t to each and switched their SQLite handles to SQLITE_OPEN_FULLMUTEX. create_record, put_record, delete_record, and apply_writes were each split into a _locked internal variant plus a public lock/unlock wrapper, because create_record_with_backlink_cleanup calls apply_writes_locked internally and a naive single lock would have deadlocked recursively. Read paths (describe, get_head, verify_head, get_record, export, get_blocks) got locking too, and blob_store's delete path was fixed to copy node data under the lock but do file I/O outside it, so disk latency doesn't block other threads.

That pass didn't catch everything. metalbear_blob_store_dissociate, on dropping a blob's last reference, unlocked the store mutex before calling back into the separate, re-locking metalbear_blob_store_delete — a window where a concurrent associate() on the same CID could find the not-yet-deleted node, append a fresh reference, and have the pending delete destroy it anyway. The blob would 404 on every future read even though a record still named it. Found via adversarial review of the concurrency model, not a bug report. The fix factors deletion into blob_store_delete_locked and calls it from within the same critical section that drops the refcount to zero. Reproducing it needed a specific test design: a naive sequential pthread_create gave the first thread such a head start it always finished before the second one started, so the race never fired. Holding every thread at a release gate until all are created, then releasing them together, reliably reproduced it — 5 of 64 dissociate/associate pairs failed with the bug reverted, and zero failed with the fix in place.

A separate patch (fix: resolve deadlock, admin auth, and invite code segfault) cleaned up three pre-existing test failures surfaced once real locking existed: emit_commit_event_ops called metalbear_repo_store_export, which locks s->mutex on entry, from a caller chain that already held the lock — fixed with an _locked export variant. Separately, admin_authenticated stripped trailing whitespace from the client's base64 but not from its own EVP_EncodeBlock output, which appends a trailing newline on macOS, so Basic auth comparisons failed there specifically. And wf_http_post was attaching both the client's cached Bearer token and a caller-supplied Basic auth header; since MHD serves whichever Authorization header it sees first, admin requests were arriving as Bearer instead of Basic.

the admin audit

com.atproto.admin.getAccountInfo and getInviteCodes accumulated a long tail of field gaps, closed one at a time against the reference's util.ts and getInviteCodes.ts. indexedAt was stamped with the current request time on every call instead of the account's real creation time — fixed by adding an accounts.created_at column, migrated in with an ALTER TABLE tolerant of "duplicate column name" for already-migrated databases, with existing rows honestly backfilled to the migration time itself rather than a fabricated earlier date. deactivatedAt was never read back out even though metalbear_account_deactivate/activate had tracked it all along. invitedBy and invites needed a new query joining invite_code_use to invite_code, matching on DID or handle since create_account only records the handle it saw at signup.

getInviteCodes itself emitted a fabricated availableBy field and a bare integer for uses instead of the lexicon's array of {usedBy, usedAt} records — first fixed by emitting an empty array (reasoning, incorrectly, that the registry didn't keep per-redemption history), then corrected again once it turned out invite_code_use had been populated on every consume_invite_code call the whole time, just never queried back out. Fixing that read surfaced a second bug: the query sorted by used_at alone, a second-precision timestamp that ties silently under SQLite's unspecified order for same-second redemptions, so rowid was added as a secondary sort key. Pagination was missing entirely too — getInviteCodes enumerated every account's codes into one unbounded list. It's now a single keyset-paginated query ordered by createdAt DESC/code DESC, keyset rather than offset because an offset silently skips or repeats rows as codes are created concurrently; a malformed cursor now gets an honest 400 instead of being ignored.

Other fixes in the same vein: deleteAccount now succeeds on a DID that hosts no account here rather than 404ing, matching the reference's unconditional delete and making a retried request idempotent instead of confusing. updateAccountEmail's admin override now deletes every outstanding email token for the account, and updateAccountPassword now revokes all existing sessions on a reset, reusing the same metalbear_auth_delete_all call the takedown path already relies on. Password length caps were wrong in both directions — createAccount enforced 64 characters despite a comment claiming parity with the reference's 256-character NEW_PASSWORD_MAX_LENGTH, and admin.updateAccountPassword had no cap at all. And getAccountInfo, sendEmail, and updateAccountEmail all answered an unresolvable account with 404 AccountNotFound where the reference uses 400 InvalidRequestError.

write-path parity

The repo write endpoints had accumulated their own small divergences. createRecord/putRecord/applyWrites reported a missing record as an invented RecordNotFound error; the reference uses plain InvalidRequest for every non-swap write failure, so that code path was removed. Read-side getRecord had the opposite problem — it answered a missing or CID-mismatched record with a 404 instead of the reference's 400 InvalidRequestError/RecordNotFound, at three separate call sites including the takedown guard in server.c. check_record now enforces the $type match even when validate=false, validationStatus is omitted from responses (not emitted as null) when validation was skipped, and putRecord omits its commit field on a true no-op write — all matching specific lines in the reference's prepare.ts and putRecord.ts. A missing check meant these endpoints had no request body size limit beyond blob uploads; check_body_size now caps createRecord/putRecord/applyWrites at 1MB ahead of JSON parsing, matching the reference's jsonLimit.

Two closed-issue fixes went further. fc493f3 ports the reference's explicit-slur detection into check_rkey (closing #29): patterns are mechanically extracted from the reference's explicit-slurs.ts by scripts/gen_explicit_slurs.py into a generated explicit_slurs_patterns.c rather than transcribed by hand, and matched via PCRE2 rather than POSIX regex.h because the reference's patterns use Unicode character classes POSIX can't express. 684753e rejects legacy-shaped blob refs ({cid, mimeType}, no $type) at write time across createRecord/putRecord/applyWrites, closing #30 — MetalBear tracked these shapes for blob bookkeeping but never actually rejected them the way repo/prepare.ts's enumBlobRefs does.

importRepo got the most substantial rework, closing #22 per an explicit user decision: it now adopts an imported commit verbatim — its own signature, never re-signed or re-chained — instead of reapplying the diff as a freshly-signed commit, and emits no firehose event on import, matching the reference exactly even though every other repo-mutating endpoint does emit one. Building this against the real storage layer (not just the reference's TypeScript) surfaced two bugs: wf_repo_diff_apply unconditionally frees whatever roots pointer it's handed, which crashed against s->car.roots's normal non-owning pointer, and the same function prunes blocks the diff removed — desyncing s->car from the DB blocks table, since every other write path leaves old blocks present-but-unreachable rather than pruning them. A separate, earlier commit (131a70f) gated importRepo on an accepting_imports config flag and a max_import_size cap, since any valid session could previously bulk-replace a repo with no size limit and no way for an operator to refuse imports at all.

sync, blobs, and moderation

com.atproto.sync.getRecord shipped a bare commit+leaf CAR with no MST path nodes — unverifiable against the commit's MST root — and 404'd on a missing record instead of proving its non-inclusion (closing #27). It now calls Wolfram's wf_repo_get_record_proof to walk the MST path to the key, present or not, and assembles a CAR from the commit, every proof node, and the leaf if one exists; a missing record is now represented by the CAR itself, a valid non-inclusion proof, not an error response. It was also this endpoint's first test coverage.

Blobs gained reference tracking: metalbear_blob_store_associate/_dissociate/_is_referenced maintain a per-blob set of referencing record URIs, persisted as a .refs sidecar for file-backed stores, and dropping a blob's last reference now deletes it outright rather than leaving it orphaned forever — MetalBear previously had no equivalent to the reference's deleteDereferencedBlobs. Separately, the direct blob-store registration path (metalbear_xrpc_server_register_blob_store_resolver, used by embedders rather than the stock binary) was serving raw blob bytes with only a Content-Type header, missing the X-Content-Type-Options: nosniff, Content-Disposition: attachment, and CSP headers the production sync.getBlob route always applied — a gap that would let an attacker-uploaded HTML or SVG blob execute as same-origin script against session tokens, on a code path that happened not to be reachable from a stock binary but is public API.

Smaller fixes rounded out this subsystem: createReport now includes the union $type and params fields it was missing; local getProfile responses get avatar, banner, and labels overlaid from the AppView instead of returning bare local state; and the proxied-URI buffer for app.bsky.* calls was widened from 1024 to 8192 bytes after notification queries with long cursors started tripping spurious 414 UriTooLong errors. Separately, authenticate() decoded a bearer token's unverified sub claim and passed it straight into context_for_did(server, sub)->auth without checking for NULL — any JWT-shaped token naming an unknown DID crashed the whole multi-tenant server before signature verification ever ran, reachable by anyone. The identical pattern in proxy_fallback already checked for NULL first; authenticate() just hadn't followed its own codebase's established pattern.

carving server.c apart

server.c had grown to 3781 lines and repo_store.c to 3619, each bundling unrelated concerns. repo_store.c split into repo_store.c (the storage engine, unchanged in scope), repo_routes.c (the com.atproto.repo.* XRPC handlers), and did_document.c (DID document generation) — the handler section had been interrupted mid-file by nine core engine functions, which took a second pass to get right. server.c shed its operator-facing surface (/metrics, /_debug/health, the landing page, and their shared string helpers) into src/ops/status.c, then describeServer, /operator.json, and _health into src/ops/ops_routes.c, then account-lifecycle handlers (requestAccountDelete, deleteAccount), getActorPreferences/putActorPreferences, and the 13 com.atproto.admin.* handlers into their respective domain files. The wider src/*.c tree was reorganized the same way — repo storage, OAuth, account resolution, DNS handle publishing, moderation, and ops tooling each got their own subdirectory, leaving only server.c, sequencer.c, email.c, and main.c at the root as genuinely cross-cutting files.

Two smaller additions round out the server's operational posture from this stretch: the sequencer now sends ConsumerTooSlow (declared but previously unused on subscribeRepos) before dropping a subscriber whose outbound buffer overflowed, instead of dropping silently; and repo writes, updateHandle, and sync.getRepo gained rate-limit buckets they'd had none of before — repo-write-hour/repo-write-day weighted by write cost (create=3, update=2, delete=1, matching the reference's calcPoints), keyed by DID.