Hi everyone, I’m Sreehari, also known online as owlpharoah (op3kay on Matrix). I’m a second year student at IIIT Jabalpur. This summer I worked on the foundations of a GraphQL server in Rust that sits over the MusicBrainz PostgreSQL database, under the guidance of @bitmap and @jadedblueeyes.
The Setting
MusicBrainz already has an XML/JSON API, but getting related data out of it means chaining together inc parameters, and browsing support differs from one entity type to the next. Looking up five artists at once isn’t really possible either.
GraphQL fixes most of this by letting a client ask for exactly the fields and relationships it wants in one query. It also lets the server check how expensive a query is before running it, instead of finding out after the database has already taken the hit.
Before writing the proposal, I built a rough prototype covering Artist, Release Group, Release, and Recording, mainly to see how things would click together. Two problems showed up: N+1 queries on relationship fields, and the fact that depth limiting alone doesn’t catch a shallow query that’s still expensive. Both became core parts of the proposal.
The Plan
The proposal scoped the project to six entity types: Artist, Release Group, Release, Recording, Label, and Area. Each would be queryable by MBID, with the usual relationships between them, plus aliases, tags, genres, and ratings across the board.
A few decisions were made initially:
- DataLoaders would follow a two tier split. One loader maps an entity’s internal id to the hydrated entity itself and gets reused everywhere that entity shows up. A separate, thinner loader maps a parent id to a list of child ids.
- Fields that need a loader call would live behind ComplexObject, so they only run when a client actually asks for them.
- Pagination would use keyset pagination instead of offset pagination, since offset pagination gets slow and inconsistent on large tables that change often.
- Query safety would come from depth limiting plus a complexity weight on every resolver.
key goals
- A working GraphQL server covering the six entity types
- Schema level depth limiting and query cost analysis
- A performance baseline from load testing.
The Result
DataLoader infrastructure is in place across all six entities, following the two tier split. Loaders exist for tags, ratings, artist credit, genres, annotations, aliases, ISNI and IPI identifiers, and MBID to internal id resolution. Hydration loaders are shared across every relationship that points at a given entity instead of duplicated per relationship.
MBID redirect handling lives inside each loader’s load function. When a primary table lookup misses, the unresolved MBIDs get batch queried against the matching *_gid_redirect table, and any hits get merged into the result map. A resolver further up never has to know a redirect happened.
Keyset pagination runs across the paginated fields using ROW_NUMBER() OVER (PARTITION BY parent_id ORDER BY child_id) in a single batched query, so one query can apply a per parent limit across a whole batch of parents. The cursor ended up as a plain integer rather than the opaque string from the proposal.
Query complexity weights reflect actual database work: a scalar field costs its default, a single hop DataLoader field costs a flat amount, a paginated one to many field scales with the requested page size, and multi hop fields carry a multiplier on top.
Integration tests cover all six entities.
Week 7’s load testing with k6 turned up two findings worth fixing. There was an N+1 on the isrc field, fixed with a dedicated RecordingIsrcLoader, and a gap in the complexity limiter where a pathological query executed instead of getting rejected outright.
On the infrastructure side, CI runs pre commit hooks and no longer has dead code warnings, and documentation is wired up through Magidoc in Docker compose.
What I Learned
The two tier loader split sounded simple on paper, but it’s saved me a lot of time in practice. Adding a new relationship is now mostly copying hydration logic that already works, instead of writing it fresh.
Fixture data needs to be checked against the database it’s running against, not assumed to be stable. musicbrainz-docker’s sample dumps import non-deterministic subsets of the data, so an MBID that resolves cleanly on my machine can point at something else, or nothing, on someone else’s. That cost me a few confused debugging sessions before I figured out what was going on.
What’s Next
- Criterion benchmarking, to compare the current per row queries on ComplexObject fields like Release.date against a batched loader variant, and to compare the two hop ArtistCredit and Tags loader patterns against a single joined loader.
- Moka caching is still an open evaluation.
- Extended entity coverage beyond the original six.
Conclusion
This was an awesome summer and i enjoyed thinking about and wiring up the API schema and the Postgresql database, fixing bugs, and everything in between. It was really satisfying watching the two tier loader pattern click into place once and then just work for every relationship added after it.
Thanks to my mentors for the guidance, especially on the DataLoader architecture, which I wouldn’t have landed on alone. And thanks to the wider MetaBrainz community for the space to build this in. It’s been a pretty nice summer of query plans, a lot of Rust, and debugging, and I’d do it again.