Category: NGS

MusicBrainz Next Generation Schema (or simply NGS) refers to the updated database schema released on 18 May 2011.

MusicBrainz NGS release date: May 16th

I’m pleased to announce that we’re finally confident enough with the NGS codebase that we’re ready to commit to a release date: May 16th! There are a tremendous number of things that we still need to finish, but we’re working hard to meet this deadline.

Two weeks prior to the deadline we will begin the process to prepare our users for the switchover. We expect to have about a two week period where we are going to discourage new edits and encourage people to vote on existing edits. We anticipate not allowing any new edits to be submitted by May 9th and we hope to encourage the community to help us review all pending edits in the last week. It would be nice to not have any open edits before we begin the migration to NGS, since we will need to close any open edits at that time. All of these dates are still in flux — we will have a complete migration schedule on May 2nd.

Tomorrow we will be releasing new NGS data, start NGS replication over and hopefully also have a new NGS Virtual Machine for you to download. Stay tuned!

Update: We had previously agreed on a 7 days no-new-edit timeframe; this post now reflects this.

MusicBrainz Next Generation Schema: Release Candidate 2!

I’m pleased to announce that the MusicBrainz dev team just completed the Release Candiate 2 release of the MusicBrainz Server Next Generation Schema!

We’ve loaded the latest data (2011-3-19) and converted it to NGS and loaded it onto our test server — please come take a look! Also note: This is going to be the last official release before we push NGS live! Please take a look at NGS and tell us about things that you think will not work. This may be your last chance before it gets real! Any bugs/problems you find, please report them to our bug tracker and mark the bugs for the fix version of NGS Final.

To see the list of issues/bugs that we’ve closed for this release, please take a look at our list of closed bugs for RC2.

We’ve turned on the EditBot on this server and it will send you mail if you give it your email address. We have reset all of the email addresses today, so if you’ve given the test server your email before and confirmed it, you will need to do that again. Sorry!

We had one minor data migration piece that wasn’t completed before the RC2 deadline, so we’re actually going to convert and re-load the data one more time in a couple of days. Once that happens, we will turn on the NGS replication stream again, but for the next 2 days there will be no replication packets coming from the test server.

Also, next week we will load NGS onto our newly purchased hardware so we can do some actual load testing to make sure the new site doesn’t tip over when we switch over. This should be really exciting to see! 🙂

Thanks to everyone who worked on this release! We’re all getting excited about NGS coming soon!

MusicBrainz Next Generation Schema Release Candidate 2 will be released on March 21

We’ve finally whittled down our open bugs and various user interface issues that we’re now confident to say that NGS RC2 will be released on March 21st.

We’re finishing on working on the current set of bugs for RC2 right now. Any new bugs should be added under the new NGS – Release Version in Jira.

Shortly after March 21st we will begin testing NGS on our new server hardware that we will be buying as a result of our fundraiser. This will give us some real live testing opportunities compared to testing on our puny test server.

New NGS Virtual Machine available

The promised Virtual Machine for MusicBrainz Next Generation Schema Release Candidate 1+ is now available for download. This virtual machine is setup as a slave and can pull replication data from the test server by running the admin/replication/LoadReplicationChanges from the musicbrainz-server codebase about 10 minutes after every hour.

See our wiki page for details on how to use this VM

Fundraiser for NGS Hardware

Rolling out NGS (Next Generation Schema) onto the main servers is going to be hard if we have to do it with our current hardware. If we can raise $15,000 to purchase new servers we will make this process much easier and with much less down time. Also, our current hardware is starting to get old and more failure prone. By purchasing newer hardware we can serve more users using less power, which will make MusicBrainz be responsible for less pollution!

Please make a donation today to help MusicBrainz raise money to cover the costs for a much needed hardware upgrade:

Our Next Generation Schema, improves MusicBrainz on many fronts. Its a complete re-write of our 11 year old codebase using modern tools (Catalyst, Moose, Template Toolkit) and a drastic improvement of our schema. The new schema fixes many of the problems of the old schema and introduces many new concepts that allow us to model music data better.

For instance, instead of having conflated artist names like “Queen & David Bowie” we will now have Artist Credits, which will link to the artists “Queen” and “David Bowie” individually. Releases will now have mediums and tracklists which will allow us to re-use tracklists in different releases, which makes release more accurate and removes duplication from the database. Tracks have been changed to recordings and if the same recording is used in two different releases the same recording (with the same MBID) will be used in both releases. NGS also introduces new concepts like musical Works, that can represent Beethoven’s 5th Symphony as an abstract Work, which then has been recorded into recordings and releases after his death. The goal is to remove artists like Beethoven and Bach from the list of releases and recordings since they never actually recorded anything while they were alive.

The overall goal is to allow MusicBrainz to grow into a complete music encyclopedia, rather than a CD lookup service that it started out as. This new codebase and schema allows us to grow and add many more features that we’ve not been able to add to the old code base. If you’d like a preview, please take a look at our test server.

All donations are tax deductible since MetaBrainz is a 501(c)3 non-profit! Thank you for your support!

Virtual Machine for NGS RC1 available for download

I’ve just finished uploading the Virtual Machine for MusicBrainz Next Generation Schema Release Candidate 1. This virtual machine is setup as a slave and can pull replication data from the test server by running the admin/replication/LoadReplicationChanges from the musicbrainz-server codebase about 10 minutes after every hour.

This is an easy way to jump in and take a look at how NGS and replication in NGS will work! (hint: the replication doesn’t change much)

UPDATE: See our wiki page for details on how to use this VM

MusicBrainz NGS RC1 release: A quick update

After a great day of teamwork in preparing for today’s RC1 release it was the database that wouldn’t play along. Our code is ready, but we’re hitting glitches on the database on our new test server.

We’re now waiting for a new clean migration of the data which takes several hours, so we need to postpone the RC1 release for a few hours. 🙁

We’re working hard to get RC1 out ASAP — stay tuned!

Next Generation Schema Release Candidate 1: Monday January 17th

As of today we can finally see the light at the end of the tunnel! We feel confident that we can finish all new features (and most improvements as listed in jira) by January 17th. Once we reach RC1 we’re going to freeze the features for NGS and only work to fix bugs in our codebase.

That said, we hope to release NGS onto the main servers sometime in February.

Its been a very long road to NGS, but it is finally tangibly close. I’m getting excited!

Next Generation public schema frozen & new data dump available

Hello and Happy New Year!

Very late last year we finally hit a milestone we’ve been working towards for quite some time: Freezing the public NGS schema!

The public schema refers to the collection of tables that will be replicated in NGS. See below for a complete list of all those tables. However, we may make small changes to some of the tables in NGS that are not replicated. This should not affect anyone, unless you are actively hacking on the MusicBrainz Server code.

Also, we have a new data dump for this frozen schema available for anyone who would like to play with it. This data dump will cleanly import into the current version of the server source code in git.

All the tables that will be replicated for NGS:

annotation, artist, artist_alias, artist_annotation, artist_meta, artist_tag, artist_credit, artist_credit_name, artist_gid_redirect, artist_name, artist_type, cdtoc, clientversion, country, gender, isrc, l_artist_artist, l_artist_label, l_artist_recording, l_artist_release, l_artist_release_group, l_artist_url, l_artist_work, l_label_label, l_label_recording, l_label_release, l_label_release_group, l_label_url, l_label_work, l_recording_recording, l_recording_release, l_recording_release_group, l_recording_url, l_recording_work, l_release_release, l_release_release_group, l_release_url, l_release_work, l_release_group_release_group, l_release_group_url, l_release_group_work, l_url_url, l_url_work, l_work_work, label, label_alias, label_annotation, label_meta, label_gid_redirect, label_name, label_tag, label_type, language, link, link_attribute, link_attribute_type, link_type, link_type_attribute_type, medium, medium_cdtoc, medium_format, puid, recording, recording_annotation, recording_meta, recording_gid_redirect, recording_puid, recording_tag, release, release_annotation, release_gid_redirect, release_meta, release_label, release_packaging, release_status, release_group, release_group_annotation, release_group_gid_redirect, release_group_meta, release_group_tag, release_group_type, release_name, replication_control, script, script_language, tag, tag_relation, track, track_name, tracklist, tracklist_index, url, work, work_annotation, work_gid_redirect, work_meta, work_name, work_tag, work_type