2012-01-12 Schema change upgrade instructions for replicated setups

We just finished the latest schema change release! If you are running a full musicbrainz-server mirror, then follow these instructions for upgrading your installation:

  1. Take down the web server running MusicBrainz, if you’re running a web server.
  2. Turn off cron jobs if you are automatically updating the database via cron jobs.
  3. Make sure your REPLICATION_TYPE setting is RT_SLAVE
  4. Switch to the new code with git fetch origin followed by git checkout v-2012-01-12-schema-change-2
  5. Install any necessary perl modules by running perl ./Makefile.PL and sudo make installdeps
  6. Run ./upgrade.sh from the top of the source directory.
  7. Set DB_SCHEMA_SEQUENCE to 14 in lib/DBDefs.pm
  8. Turn cron jobs back on, if needed.
  9. Restart the MusicBrainz web server, if needed.

If you are running a mbslave mirror, check out the latest code and read the upgrade instructions in the README file.

Schema change, 2012-01-12

We’ve just finished pushing out the scheduled schema change update today. A slightly rocky experience, but we’ve managed to get everything ironed out now and our servers are back to their normal selves. Before, I get into release notes, a few notes for people who’d like to upgrade their non-replicated servers. For replicated servers, please see this post.

  1. First, update your repository to the v-2012-01-12-schema-change tag.
  2. Next, disable any MusicBrainz cron jobs you might have scheduled to run
  3. Run ./upgrade.sh provided in the root directory of your musicbrainz-server checkout.
  4. Update your DBDefs DB_SCHEMA_SEQUENCE configuration option to 14.
  5. Turn any cron jobs back on

And onto the release notes!

Bug

  • [MBS-2121] – Deleting track/row in advanced tracklist editor while artist credits "dialog" is open leaves it stuck
  • [MBS-2390] – It’s not really possible to assign a track to a recording because cannot see length of track in either tracklist or proposal list
  • [MBS-2620] – Most ‘edit’ edits do not check for conflicts
  • [MBS-3393] – App keeps DB connections open and doesn’t reconnect upon DB restart.
  • [MBS-3696] – Using Jamendo as License Relationship fails due to Javascript change to Cover Art relationship
  • [MBS-3972] – ISWC column on works page should be nowrap
  • [MBS-4046] – Releases with pending edits are not marked as such in label page
  • [MBS-4099] – The server should give a good error message when it can’t connect to the DB
  • [MBS-4103] – Problems building database extensions with Ubuntu 11.10
  • [MBS-4129] – "Edit artist" doesn’t change artist credit for credits that were unchecked by default but checked manually
  • [MBS-4132] – Rate limited pages send HTML as text/plain
  • [MBS-4134] – Javascript sorting broken on Country statistics page since comma has been introduced
  • [MBS-4136] – beta.mb: recording edits generated by release editor when updating recording information from track changes are not auto-edits
  • [MBS-4138] – Language-Script page in statistics broke
  • [MBS-4146] – Track times deleted if using tab key
  • [MBS-4149] – The added referrer stuff breaks Amazon UK links
  • [MBS-4157] – beta.mb: Edit recording edits seem to create a ‘edit release’ edit too
  • [MBS-4169] – CDToc page shows blank/empty catalog numbers separated by commas

Improvement

  • [MBS-1482] – Report: no language, no script
  • [MBS-2439] – Releases with multiple catalog numbers are shown multiple times on the Label page.
  • [MBS-2513] – Allow updating recording information when track information changes via the release editor
  • [MBS-3288] – Release group type isn’t shown when looking up a release group
  • [MBS-4105] – Warn when merging releases with track artists that don’t match

New Feature

  • [MBS-4147] – Display AcoustIDs using JavaScript

Task

  • [MBS-3672] – Remove the deprecated Live Sound Engineer relationship

Sub-task

  • [MBS-2676] – Filter by relationship type

Schema change releases for 2012

One of the issues raised at the last summit was that our customers could use more time to prepare for schema change releases, since they require engineering effort on their part. In an effort to meet our customer’s needs we’re going to a set schedule for schema change releases. Going forward we’re going to have two schema change releases per year: On or about 15 May and on or about 15 October.

We’ve picked these two dates as dates that have the least amount of impact from holidays and people taking holidays. Most companies have normal working schedules around these dates, which should allow companies to dedicate the required resources to handle our schema changes.

However, we have one significant change that we need to push out on a more timely basis than May of next year. For that reason we’re going to plan a one-time exception to our new schedule for 12 January 2012. You can see the tickets we’ve scheduled for release in January in this schema change milestone.

We’ve already created release versions in Jira for all of the schema change releases in 2012. As we go through the year we’re going to add tickets to that milestone. Of course, we’re going to make lots of noise as these schema change release dates approach. We’re going to post a list of tickets that will be included no later than a month before the release.

Any questions? Ask them in the comments!

Schema change release, 2011-07-11

Today we released our first schema change update since NGS. This change is quite a radical one, as it merges both of our databases (“READWRITE” and “RAWDATA”) into a single database. For most users of the database, this probably won’t affect you, but you’re encouraged to run the upgrade process anyway. Here’s what you need to do:

  1. Take down the web server running MusicBrainz, if you’re running a web server.
  2. Turn off cron jobs if you are automatically updating the database via cron jobs.
  3. Set DB_SCHEMA_SEQUENCE to 13 in lib/DBDefs.pm
  4. Make sure your REPLICATION_TYPE setting is RT_SLAVE
  5. Switch to the new code with git fetch origin followed by git checkout v-20110711-schema-change
  6. Run ./upgrade.sh from the top of the source directory.
  7. Install the perl modules Algorithm::Merge and Algorithm::Diff
  8. Turn cron jobs back on, if needed.
  9. Restart the MusicBrainz web server, if needed.

This process may take a while, as it has to dump one database into another, and download a few extra changes to ensure slaves aren’t missing any data. The RAWDATA database should no longer be in use and you should be able to drop it, but waiting to see that everything is working well might be a good idea.

This schema change does not introduce any new data. For everyone else, here’s a list of what got fixed since the last release!

Bug

  • [MBS-1977] – ModBot is unable to close some edits
  • [MBS-1979] – Unable to edit a “later translated versions” relationship with change direction
  • [MBS-2026] – Subscribed artists open edits won’t load for editors with large amounts of subscriptions
  • [MBS-2442] – MB postgres unaccent extension overwrites unaccent.so library shipped with postgres-contrib
  • [MBS-2689] – Timeline isn’t working in Opera 10
  • [MBS-2698] – Some tables are not replicated through Live data feed
  • [MBS-2812] – Adding track times has caused a failed dependency for an edit which changed the track titles.
  • [MBS-2826] – Web service returns malformed XML (not escaped properly)
  • [MBS-2831] – “Edit medium” edit display: Artist credits changes are incorrect
  • [MBS-2974] – I don’t receive e-mail when someone votes no to my edit any more
  • [MBS-2981] – Changing the case of recording comments should be auto-edits
  • [MBS-2986] – tracklist_index was not populated during NGS
  • [MBS-2995] – Caught exception in MusicBrainz::Server::Controller::WS::2::ReleaseGroup->release_group_browse “Can’t call method “format”

Improvement

  • [MBS-1500] – Remove tracklist_index.tracks
  • [MBS-1707] – Advanced tracklist in RE: “Title” to “Disc title”
  • [MBS-2242] – Disable editing of Medium title when there’s only one medium
  • [MBS-2434] – Combine READWRITE and RAWDATA
  • [MBS-2462] – Other edit types that don’t highlight what has changed between old and new values
  • [MBS-2583] – “Edit Medium” should show you which medium is being edited with respect to the overall release
  • [MBS-2767] – Release-group XML result should include first release date

Next Generation Schema Release Candidate 1: Monday January 17th

As of today we can finally see the light at the end of the tunnel! We feel confident that we can finish all new features (and most improvements as listed in jira) by January 17th. Once we reach RC1 we’re going to freeze the features for NGS and only work to fix bugs in our codebase.

That said, we hope to release NGS onto the main servers sometime in February.

Its been a very long road to NGS, but it is finally tangibly close. I’m getting excited!