New NGS dataset, new VM and replication started

We have a new NGS dataset available. This data set contains no known bugs (at this point in time) and is the basis for our restarted replication stream.

If you’re interesting in testing the NGS replication stream, please download and install this data set and then run admin/replication/LoadReplicationChanges to pull in the latest changes.

I’ve also uploaded a new NGS Virtual Machine — for details on how to use this virtual machine, please see the wiki page for the virtual machine. This VM is also setup to start replicating off this NGS data stream. Invoke the replication mechanism as mentioned above.

At this point we’re not certain if this is the last time we’re going to start the replication or if we’re going to do it one more time before we release NGS. Stay tuned!

Dear Anonymous web service user: We're seeing too much of you!

If you’ve installed an application and you’re getting an excessive number of 503 errors for this application, its because the application author was not a good citizen. We’ve throttled your traffic until you can behave better. Please contact the author of the application and let them know about this blog post!

If you are the author of this application, please post a comment so we can start a dialog!

Reduced editing phase has begun!

We’ve started our reduced editing phase, where everyone is encouraged to not add any new edits to MusicBrainz until we release NGS. We’ve added ugly banners to each edit submission page and the home page on musicbrainz.org to remind everyone that they should be voting more than editing this week.

Next week we’re going to disallow edits entirely. For more information on these pre-NGS changes, please read our Preparing for NGS guide.

Lets get the edit queue down as fast as we can!

New NGS data, minor schema change and VM available!

(After fixing one buggy dataset, we have a couple of confirmations that the updated data set is ready to go.)

We have a new NGS dataset available — this data set still has a couple of migration bugs for ARs. The “RecordingA is cover of RecordingB” relationships are borked, but the rest of the data looks to be OK.

This new data also contains one very minor schema change: In the artist_credit_name table, the join_phrase column went from VARCHAR(32) to TEXT, to allow longer join phrases. This shouldn’t really affect anyone, but it is something to be aware of if you’re expecting a VARCHAR(32).

We’re going to work on fixing this bug and then we will put out a new data set and start replication off the new servers. In the mean time, feel free to play with the new data.

I’ve also uploaded a new NGS Virtual Machine — for details on how to use this virtual machine, please see the wiki page for the virtual machine.

UPDATE 1: Fixed the dataset and updated the link.
UPDATE 2: Added a note about the schema change.

MusicBrainz NGS release date: May 16th

I’m pleased to announce that we’re finally confident enough with the NGS codebase that we’re ready to commit to a release date: May 16th! There are a tremendous number of things that we still need to finish, but we’re working hard to meet this deadline.

Two weeks prior to the deadline we will begin the process to prepare our users for the switchover. We expect to have about a two week period where we are going to discourage new edits and encourage people to vote on existing edits. We anticipate not allowing any new edits to be submitted by May 9th and we hope to encourage the community to help us review all pending edits in the last week. It would be nice to not have any open edits before we begin the migration to NGS, since we will need to close any open edits at that time. All of these dates are still in flux — we will have a complete migration schedule on May 2nd.

Tomorrow we will be releasing new NGS data, start NGS replication over and hopefully also have a new NGS Virtual Machine for you to download. Stay tuned!

Update: We had previously agreed on a 7 days no-new-edit timeframe; this post now reflects this.

MusicBrainz Next Generation Schema: Release Candidate 2!

I’m pleased to announce that the MusicBrainz dev team just completed the Release Candiate 2 release of the MusicBrainz Server Next Generation Schema!

We’ve loaded the latest data (2011-3-19) and converted it to NGS and loaded it onto our test server — please come take a look! Also note: This is going to be the last official release before we push NGS live! Please take a look at NGS and tell us about things that you think will not work. This may be your last chance before it gets real! Any bugs/problems you find, please report them to our bug tracker and mark the bugs for the fix version of NGS Final.

To see the list of issues/bugs that we’ve closed for this release, please take a look at our list of closed bugs for RC2.

We’ve turned on the EditBot on this server and it will send you mail if you give it your email address. We have reset all of the email addresses today, so if you’ve given the test server your email before and confirmed it, you will need to do that again. Sorry!

We had one minor data migration piece that wasn’t completed before the RC2 deadline, so we’re actually going to convert and re-load the data one more time in a couple of days. Once that happens, we will turn on the NGS replication stream again, but for the next 2 days there will be no replication packets coming from the test server.

Also, next week we will load NGS onto our newly purchased hardware so we can do some actual load testing to make sure the new site doesn’t tip over when we switch over. This should be really exciting to see! 🙂

Thanks to everyone who worked on this release! We’re all getting excited about NGS coming soon!

Our hardware fundraiser is complete: We raised $15,527.50

After pestering people for a month, I’m pleased to announce that we’ve successfully closed our fundraiser. We’ve exceeded our goal by raising $15,527.50!

The following companies & individuals contributed:

$5000.00 Google
$3708.50 Individual user donations
$1389.00 SoundCloud
$2000.00 EchoNest
$1000.00 Matt Mullenweg (WordPress)
$1000.00 Decibel.net
$500.00 Magic MP3 Tagger
$500.00 Grooveshark
$300.00 Songkick
$130.00 Affinity Chiropractic (the people MetaBrainz rents an office from)
$15,527.50 in total!

I will send out thank you mails to our 83 individual donors later today! Thanks to everyone who contributed! I’ve already ordered the hardware we wanted and will start installing it this week. Hopefully next week we can start loading NGS on this new hardware.

Picard 0.13 binary releases now available for Linux, Mac and Windows

Picard 0.13 was recently released and thanks to the packaging efforts of Michael Wiencek we now have a working Mac DMG for Intel Macs running OS X 10.4+! At this point we do not have any support for older PPC Macs and it doesn’t look like we will ever have support for that. Sorry. There is an older 0.11 Universal Binary that PPC users can use.

Download your Picard version 0.13 binary version now!

Thanks to Lukáš, Philip, Michael and everyone else who worked on this release!

UPDATE: You see a change log for this release here.