Category: Breaking Changes

Schema change server update, 2012-05-15

Nearly one year after we released NGS, we have another schema change update with lots of new features!

This release contains 9 new features and improvements that take advantage of the new schema. These are:

  • More social user profiles which can now have Gravatars, languages (and the users proficiency) age and country.
  • More expressive aliases for artists, labels and works. Aliases can now have types, sort names and multiple aliases may be used per a locale, along with the ability to mark one alias as ‘primary’ for that locale.
  • Release group types have been separated into primary and secondary types. A release group now has 1 primary type and may have multiple secondary types. This allows us to have ‘remix compilation albums’, for example
  • Works may have multiple ISWCs
  • Artists, labels and relationships may be marked as ‘ended’ to indicate that they have ended, but the exact date is not known
  • Vinyl style/free text track numbers are now supported.
  • Works may have a lyrics language associated with them
  • Artists and labels may have multiple IPIs
  • We have moved to use ISO 639-3 for our language table. While not all languages are exposed at the moment, this gives us a lot more flexibility going forward.

Many thanks to nikki for going way beyond our expectations for testing (and patience!); to Ian McEwen for his continued work on statistics; and to the MusicBrainz team for making this all happen.

If you have a replicated instance of MusicBrainz, please follow these instructions to get your server running on the new schema:

  1. Take down the web server running MusicBrainz, if you’re running a web server.
  2. Turn off cron jobs if you are automatically updating the database via cron jobs.
  3. Make sure your REPLICATION_TYPE setting is RT_SLAVE
  4. Switch to the new code with git fetch origin followed by git checkout v-2012-05-15-schema-change
  5. Run carton install --deployment. If you have not switched your installation to using carton, please read INSTALL.md on how to do this.
  6. Run carton exec -- ./upgrade.sh from the top of the source directory.
  7. Set DB_SCHEMA_SEQUENCE to 15 in lib/DBDefs.pm
  8. Turn cron jobs back on, if needed.
  9. Restart the MusicBrainz web server, if needed.

If you are running a mbslave mirror, check out the latest code and read the upgrade instructions in the README file.

Bug

  • [MBS-3189] – Remove unused ref_count column and related functions
  • [MBS-4616] – Add work language statistics
  • [MBS-4629] – /cover-art page shows no collections
  • [MBS-4637] – Timeline graph won’t graph anything without an entry in statistics/view.js
  • [MBS-4640] – Clicking cover art opens box with “����” (4 U+FFFD)
  • [MBS-4642] – Thickbox CSS interferes with MB CSS
  • [MBS-4647] – Cover art page allows submitting edit with no cover art when JS is off
  • [MBS-4648] – Changing cover art type from “other” to unset causes Internal Server Error
  • [MBS-4678] – upgrade.sh is not ready for testing
  • [MBS-4679] – Internal server error adding secondary types to a release

Improvement

  • [MBS-1485] – Alias types
  • [MBS-1798] – Lyrics language for works
  • [MBS-1799] – Add ISO 639-3 language codes to the database
  • [MBS-1981] – Add blog feed to the home page
  • [MBS-2240] – Aliases: certain locale can be used only once in the list of aliases
  • [MBS-2532] – Allow more than one IPI per artist
  • [MBS-2851] – Timeline graph events should be in the database
  • [MBS-2885] – Allow more than one ISWC per work
  • [MBS-3646] – Split release group attributes into two types
  • [MBS-3788] – Alias improvements
  • [MBS-4625] – Improve wording of cover art tab when cover art comes from relationships
  • [MBS-4676] – Do not allow people entering deprecated relationships

New Feature

  • [MBS-842] – Allow vinyl style track numbers and sides
  • [MBS-1385] – Support unknown end dates
  • [MBS-3704] – Allow adding sort names to artist aliases
  • [MBS-4337] – Make user profile more social: add (optional) fields avatar, gender, birth year, country

Next schema change release: May 15

We’ve talked a bit about our upcoming schema change release, but we hadn’t nailed down the exact date of the release. Now that we’re tangibly close, we’ve settled on the May 15th as the actual release date.

As a reminder, here are the tickets that will change our schema and here are all the tickets scheduled for the 15th.

Official schema change release announcement for May 2012 release

We’ve been working on hammering out the details of the upcoming schema change release and we’ve settled on 11 tickets that we’re going to implement. For a detailed description of how our database will change, please refer to the our documentation for these tickets.

This release will happen on or about 15 May, 2012.

Schema change update

As per my previous blog post, today is the deadline for submitting tickets for consideration in the May 15 2012 schema change release. Eleven tickets have been championed for this release — you can see each of these tickets in the schema change release fix version in jira.

I would like to call your attention to three tickets that I think ought to get more input from the community:

  1. Split release group attributes into two types
  2. Lyrics language for works
  3. Add ISO 639-3 language codes to the database

If any of these issues interest you, please take a look and leave a comment if you have any input. Finally, I will be keeping track of the progress of the schema change release on this wiki page.

I’m hoping to put together a complete schema change document that outlines the exact changes to our schema on April 2.

Many thanks to all of the volunteers who adopted tickets and are moving them forward!

Preparing for the May 15th schema change release

For our last schema change release we had a ton of issues around which tickets we should address and which ones were properly defined for us to work on. I’d like to make this a lot more clear for the next go round; here is what we’re going to do:

Starting today and for the next two weeks, we’re going to seek people to be the champion (sponsor) of a ticket. If you feel strongly about a schema change ticket getting taken care of, you should consider championing this ticket. Once you’ve decided to do adopt a ticket, you should assign the ticket to yourself.

Then, over the next two weeks it will be up to you to do the following:

  1. Derive consensus around the core concept of the ticket. If you go through the process of working up a ticket, but no one agrees with what you’re proposing, you’ve wasted your time. Make sure that you get buy in from others in the community. For instance, if Nikki doesn’t like it, chances are its not going to fly. 🙂
  2. Ensure that the ticket clearly states what needs to be done to implement the ticket. The ticket should essentially become or link to a requirements document. This requirements document should explain what the new feature should do. It should not explain how it should be done — we should leave the how to our developers who are going to implement the feature.
  3. Provide as much supporting documentation as you can. Mock-ups for UIs are deeply appreciated (even if they delve into the how realm of things) and very useful for meaningfully discussing these tickets.
  4. Have the ticket reviewed by a developer for clarity and completeness, then address any issues said developer may raise.

On 19 March, we’re going to look at the list of tickets that people have taken on and choose the ones that are clear enough to move forward. If you’ve done all the work outlined above, the chances are good that your ticket will be chosen to move forward. If your ticket is chosen to move forward, there will be more questions that the developers will raise — hopefully those can be tackled in the space of a week. After that we will take all of the well defined tickets and schedule them for implementation. All the other tickets that are not clear to implement will be rejected and will have to make another pass though this process in the autumn.

If you’re still interested, here is the list of schema change tickets that should be considered for this.

2012-01-12 Schema change fix for replicated slaves

Sadly, our testing for replication failed to catch a problem with the instructions we posted yesterday. If, when running the replication scripts you encountered the below error, please read on!

Attribute (conn) does not pass the type constraint because: Validation
failed for 'DBIx::Connector' with value DBI::db=HASH(0x9e957d0)
(not isa DBIx::Connector)

We made some changes to our database connection routines recently, and the replication scripts were not correctly migrated. We’ve got these changes in now though, so to fix this problem, repeat the instructions in our previous post, but use the v-2012-01-12-schema-change-2 tag. This should correspond to commit eb89c2b51f79..., which you can verify by running git rev-parse HEAD.

Sorry about these problems!

2012-01-12 Schema change upgrade instructions for replicated setups

We just finished the latest schema change release! If you are running a full musicbrainz-server mirror, then follow these instructions for upgrading your installation:

  1. Take down the web server running MusicBrainz, if you’re running a web server.
  2. Turn off cron jobs if you are automatically updating the database via cron jobs.
  3. Make sure your REPLICATION_TYPE setting is RT_SLAVE
  4. Switch to the new code with git fetch origin followed by git checkout v-2012-01-12-schema-change-2
  5. Install any necessary perl modules by running perl ./Makefile.PL and sudo make installdeps
  6. Run ./upgrade.sh from the top of the source directory.
  7. Set DB_SCHEMA_SEQUENCE to 14 in lib/DBDefs.pm
  8. Turn cron jobs back on, if needed.
  9. Restart the MusicBrainz web server, if needed.

If you are running a mbslave mirror, check out the latest code and read the upgrade instructions in the README file.

Schema change, 2012-01-12

We’ve just finished pushing out the scheduled schema change update today. A slightly rocky experience, but we’ve managed to get everything ironed out now and our servers are back to their normal selves. Before, I get into release notes, a few notes for people who’d like to upgrade their non-replicated servers. For replicated servers, please see this post.

  1. First, update your repository to the v-2012-01-12-schema-change tag.
  2. Next, disable any MusicBrainz cron jobs you might have scheduled to run
  3. Run ./upgrade.sh provided in the root directory of your musicbrainz-server checkout.
  4. Update your DBDefs DB_SCHEMA_SEQUENCE configuration option to 14.
  5. Turn any cron jobs back on

And onto the release notes!

Bug

  • [MBS-2121] – Deleting track/row in advanced tracklist editor while artist credits "dialog" is open leaves it stuck
  • [MBS-2390] – It’s not really possible to assign a track to a recording because cannot see length of track in either tracklist or proposal list
  • [MBS-2620] – Most ‘edit’ edits do not check for conflicts
  • [MBS-3393] – App keeps DB connections open and doesn’t reconnect upon DB restart.
  • [MBS-3696] – Using Jamendo as License Relationship fails due to Javascript change to Cover Art relationship
  • [MBS-3972] – ISWC column on works page should be nowrap
  • [MBS-4046] – Releases with pending edits are not marked as such in label page
  • [MBS-4099] – The server should give a good error message when it can’t connect to the DB
  • [MBS-4103] – Problems building database extensions with Ubuntu 11.10
  • [MBS-4129] – "Edit artist" doesn’t change artist credit for credits that were unchecked by default but checked manually
  • [MBS-4132] – Rate limited pages send HTML as text/plain
  • [MBS-4134] – Javascript sorting broken on Country statistics page since comma has been introduced
  • [MBS-4136] – beta.mb: recording edits generated by release editor when updating recording information from track changes are not auto-edits
  • [MBS-4138] – Language-Script page in statistics broke
  • [MBS-4146] – Track times deleted if using tab key
  • [MBS-4149] – The added referrer stuff breaks Amazon UK links
  • [MBS-4157] – beta.mb: Edit recording edits seem to create a ‘edit release’ edit too
  • [MBS-4169] – CDToc page shows blank/empty catalog numbers separated by commas

Improvement

  • [MBS-1482] – Report: no language, no script
  • [MBS-2439] – Releases with multiple catalog numbers are shown multiple times on the Label page.
  • [MBS-2513] – Allow updating recording information when track information changes via the release editor
  • [MBS-3288] – Release group type isn’t shown when looking up a release group
  • [MBS-4105] – Warn when merging releases with track artists that don’t match

New Feature

  • [MBS-4147] – Display AcoustIDs using JavaScript

Task

  • [MBS-3672] – Remove the deprecated Live Sound Engineer relationship

Sub-task

  • [MBS-2676] – Filter by relationship type