Next generation schema: Where we are today

Sorry for the delay in continuing my MusicBrainz Server Roadmap updates — we’ve been wrestling with some server configuration nightmares…

I’ll start by giving some background on the Next Generation Schema (NGS). People have been calling for an improved schema that can intelligently handle classical music, proper artist attribution and support for packages of releases (among many other things). In 2005 we held a summit in Germany where we laid down some groundwork for the requirements for a new schema and created a first rough draft schema. Holes were quickly poked into that schema, but this let us find the weak points and let us do a better job of designing the schema for the next attempt. The next attempt was an invite only summit held in London in 2007, where we created a schema that should be pretty close to what we’re actually going to put in place.

Since then we’ve been debating how to make NGS a reality. And to say the least, its been a real pain so far. The change from the existing schema to NGS is a very large project and will be far from trivial. For instance, do we migrate to NGS in one step or take a few steps? Can we keep our existing edit system or do we need to rewrite it from scratch? Now that we have the schema done, what should the user interface look like? How can we make it simple for those users who want to do simple data changes? How can we allow more expert users to make all sorts of detailed changes while keeping the user interface simple? What would the best tools for building a new UI be?

To answer these questions, Lukáš has been working on the NGS playground in his spare time. The NGS playground is Lukáš’ attempt to answer these questions. So far, he has answered two questions:

1. We should move to NGS in one step. Moving to NGS in multiple step steps will cause too many headaches and too much lost work. Each step would require extensive work to glue the old portions to the new portions and these glue layers would later be discarded. Overall, not a very efficient means of moving forward.
2. We need to dump the edit system and start over. We’ve learned a lot about how to do an edit system right — and the existing system won’t cut it moving forward. Its time to start over.

In the process, Lukáš is trying out a bunch of new tools to see how they fare. He’s also working in Python to see if that is the proper language to move forward with. While I haven’t run the NGS playground yet, Lukáš has the schema finished except for the Works concept, which is pretty straightforward. Also, I believe that the script to convert an existing database to the new format is also done. However, Lukáš’ work is only a playground — its an attempt to see how we can pull off the user interface for NGS. That’s NGS in a nutshell today.

In my next post I will continue this thread with thoughts on how we go from today’s mb_server and the Template Toolkit work and move towards the NGS work that Lukáš is doing. For now, if you’re a computer geek who is interested in looking at NGS, please take a look at the NGS playground. There isn’t much documentation yet — so consider this a “some assembly required” project. We’re interested to hear your comments on the playground — please feel free to post them here.