Posts related to the MusicBrainz encyclopedia in general: community, data, client, server, etc.
Murdos recently added a new report that highlights releases that have disc ids that will be discarded during the NGS data migration phase because the data is not conclusive enough to be reliably migrated. If you have some time, please help us review this data so we can save as much information when we switch over to NGS.
Thanks for creating that report Murdos!
(After fixing one buggy dataset, we have a couple of confirmations that the updated data set is ready to go.)
We have a new NGS dataset available — this data set still has a couple of migration bugs for ARs. The “RecordingA is cover of RecordingB” relationships are borked, but the rest of the data looks to be OK.
This new data also contains one very minor schema change: In the artist_credit_name table, the join_phrase column went from VARCHAR(32) to TEXT, to allow longer join phrases. This shouldn’t really affect anyone, but it is something to be aware of if you’re expecting a VARCHAR(32).
We’re going to work on fixing this bug and then we will put out a new data set and start replication off the new servers. In the mean time, feel free to play with the new data.
I’ve also uploaded a new NGS Virtual Machine — for details on how to use this virtual machine, please see the wiki page for the virtual machine.
UPDATE 1: Fixed the dataset and updated the link.
UPDATE 2: Added a note about the schema change.
I’m pleased to announce that we’re finally confident enough with the NGS codebase that we’re ready to commit to a release date: May 16th! There are a tremendous number of things that we still need to finish, but we’re working hard to meet this deadline.
Two weeks prior to the deadline we will begin the process to prepare our users for the switchover. We expect to have about a two week period where we are going to discourage new edits and encourage people to vote on existing edits. We anticipate not allowing any new edits to be submitted by May 9th and we hope to encourage the community to help us review all pending edits in the last week. It would be nice to not have any open edits before we begin the migration to NGS, since we will need to close any open edits at that time. All of these dates are still in flux — we will have a complete migration schedule on May 2nd.
Tomorrow we will be releasing new NGS data, start NGS replication over and hopefully also have a new NGS Virtual Machine for you to download. Stay tuned!
Update: We had previously agreed on a 7 days no-new-edit timeframe; this post now reflects this.
I’m pleased to announce that the MusicBrainz dev team just completed the Release Candiate 2 release of the MusicBrainz Server Next Generation Schema!
We’ve loaded the latest data (2011-3-19) and converted it to NGS and loaded it onto our test server — please come take a look! Also note: This is going to be the last official release before we push NGS live! Please take a look at NGS and tell us about things that you think will not work. This may be your last chance before it gets real! Any bugs/problems you find, please report them to our bug tracker and mark the bugs for the fix version of NGS Final.
To see the list of issues/bugs that we’ve closed for this release, please take a look at our list of closed bugs for RC2.
We’ve turned on the EditBot on this server and it will send you mail if you give it your email address. We have reset all of the email addresses today, so if you’ve given the test server your email before and confirmed it, you will need to do that again. Sorry!
We had one minor data migration piece that wasn’t completed before the RC2 deadline, so we’re actually going to convert and re-load the data one more time in a couple of days. Once that happens, we will turn on the NGS replication stream again, but for the next 2 days there will be no replication packets coming from the test server.
Also, next week we will load NGS onto our newly purchased hardware so we can do some actual load testing to make sure the new site doesn’t tip over when we switch over. This should be really exciting to see! 🙂
Thanks to everyone who worked on this release! We’re all getting excited about NGS coming soon!
Picard 0.13 was recently released and thanks to the packaging efforts of Michael Wiencek we now have a working Mac DMG for Intel Macs running OS X 10.4+! At this point we do not have any support for older PPC Macs and it doesn’t look like we will ever have support for that. Sorry. There is an older 0.11 Universal Binary that PPC users can use.
Download your Picard version 0.13 binary version now!
Thanks to Lukáš, Philip, Michael and everyone else who worked on this release!
UPDATE: You see a change log for this release here.
We’ve finally whittled down our open bugs and various user interface issues that we’re now confident to say that NGS RC2 will be released on March 21st.
We’re finishing on working on the current set of bugs for RC2 right now. Any new bugs should be added under the new NGS – Release Version in Jira.
Shortly after March 21st we will begin testing NGS on our new server hardware that we will be buying as a result of our fundraiser. This will give us some real live testing opportunities compared to testing on our puny test server.
The promised Virtual Machine for MusicBrainz Next Generation Schema Release Candidate 1+ is now available for download. This virtual machine is setup as a slave and can pull replication data from the test server by running the admin/replication/LoadReplicationChanges from the musicbrainz-server codebase about 10 minutes after every hour.
We’ve pushed out the latest and greatest NGS data set (2011-02-22) and updated the test.musicbrainz.org with this data and restarted the NGS replication. If you’re are testing the replication for NGS, you will need to import this new data set.
A new virtual machine with the latest NGS code and the data set is being pushed out to the FTP server now. More on that tomorrow.
Rolling out NGS (Next Generation Schema) onto the main servers is going to be hard if we have to do it with our current hardware. If we can raise $15,000 to purchase new servers we will make this process much easier and with much less down time. Also, our current hardware is starting to get old and more failure prone. By purchasing newer hardware we can serve more users using less power, which will make MusicBrainz be responsible for less pollution!
Please make a donation today to help MusicBrainz raise money to cover the costs for a much needed hardware upgrade:
Our Next Generation Schema, improves MusicBrainz on many fronts. Its a complete re-write of our 11 year old codebase using modern tools (Catalyst, Moose, Template Toolkit) and a drastic improvement of our schema. The new schema fixes many of the problems of the old schema and introduces many new concepts that allow us to model music data better.
For instance, instead of having conflated artist names like “Queen & David Bowie” we will now have Artist Credits, which will link to the artists “Queen” and “David Bowie” individually. Releases will now have mediums and tracklists which will allow us to re-use tracklists in different releases, which makes release more accurate and removes duplication from the database. Tracks have been changed to recordings and if the same recording is used in two different releases the same recording (with the same MBID) will be used in both releases. NGS also introduces new concepts like musical Works, that can represent Beethoven’s 5th Symphony as an abstract Work, which then has been recorded into recordings and releases after his death. The goal is to remove artists like Beethoven and Bach from the list of releases and recordings since they never actually recorded anything while they were alive.
The overall goal is to allow MusicBrainz to grow into a complete music encyclopedia, rather than a CD lookup service that it started out as. This new codebase and schema allows us to grow and add many more features that we’ve not been able to add to the old code base. If you’d like a preview, please take a look at our test server.
All donations are tax deductible since MetaBrainz is a 501(c)3 non-profit! Thank you for your support!
I’ve just finished uploading the Virtual Machine for MusicBrainz Next Generation Schema Release Candidate 1. This virtual machine is setup as a slave and can pull replication data from the test server by running the admin/replication/LoadReplicationChanges from the musicbrainz-server codebase about 10 minutes after every hour.
This is an easy way to jump in and take a look at how NGS and replication in NGS will work! (hint: the replication doesn’t change much)