Author: ruaok

Our forums are back!

Our forums were compromised a while ago and we had to undergo massive yak shaving in order to set up a new home for our forums. Hosting many different types of software on one server makes that server hard to administer — we felt that the proper solution was to create a Virtual server host and give each type of software a new Linux instance to live in. We started with the forums, but we’re going to be moving a lot more stuff over to this virtual host in the coming weeks/months. Hopefully we can update our blog and our wiki in this process.

In any case, the forums are back online and running with the old posts, but the latest version of PunBB. In the move we lost our customized MusicBrainz theme for PunBB — if someone feels strongly about having the theme, please take a look at the PunBB docs and create a new theme. I’ll be glad to install that new theme on our server.

Thanks and sorry the forums were offline for so long!

Post schema change fix for importing clean data

Yesterday we found a bug that prevents the import of a post schema change update data set. We’ve pushed out a fix for this and tagged it with: 

v-2012-05-15-import-fix

If you’re planning on importing a new data set, make sure to check out this tag, rather than the tag mentioned in this entry.

Search server release: 2012-05-15

In case you haven’t gotten enough of release announcements, we have another one for you. Yesterday during the main releases we also released a new search server to match the main server release. Thanks much to Paul Taylor for working on this release to be timed perfectly!

UPDATE: The search server and the MMD schema repositories have been tagged with this tag:

release-2012-05-15

Bug

  • [SEARCH-198] – The artist is getting a lowered score on MBS
  • [SEARCH-199] – Search includes empty annotations
  • [SEARCH-200] – Search on release giving to much boost to matches on CatalogNo
  • [SEARCH-201] – explain option doesnt work if search results contain non ISO-8859-1 characters
  • [SEARCH-216] – Null pointer exception when building freedb

Improvement

  • [SEARCH-157] – Be able to search for a track by its metadata OR its puid
  • [SEARCH-186] – Search Server has hard coded redirect URL
  • [SEARCH-187] – Update Junit Test from 3 to 4
  • [SEARCH-202] – Allow searching for RGs based on their releases’ status
  • [SEARCH-204] – Upgrade codebase to Lucene 3.6
  • [SEARCH-214] – Add release group ID to the web service indexed search results for recordings

New Feature

  • [SEARCH-205] – Search server should return multiple ISWCs for works
  • [SEARCH-207] – Changes due to introduction of ISO-3 language code
  • [SEARCH-208] – Chnages due to Split release group attributes into two types Schema Change
  • [SEARCH-209] – Support for Multiple IPI Artists
  • [SEARCH-211] – Support for new Track ‘Number’ field in a track
  • [SEARCH-212] – Add abiility to index, display and search works by lyrics language as part of schema change
  • [SEARCH-213] – Changes due to MBS-1385:Support unknown end dates

Task

Schema change server update, 2012-05-15

Nearly one year after we released NGS, we have another schema change update with lots of new features!

This release contains 9 new features and improvements that take advantage of the new schema. These are:

  • More social user profiles which can now have Gravatars, languages (and the users proficiency) age and country.
  • More expressive aliases for artists, labels and works. Aliases can now have types, sort names and multiple aliases may be used per a locale, along with the ability to mark one alias as ‘primary’ for that locale.
  • Release group types have been separated into primary and secondary types. A release group now has 1 primary type and may have multiple secondary types. This allows us to have ‘remix compilation albums’, for example
  • Works may have multiple ISWCs
  • Artists, labels and relationships may be marked as ‘ended’ to indicate that they have ended, but the exact date is not known
  • Vinyl style/free text track numbers are now supported.
  • Works may have a lyrics language associated with them
  • Artists and labels may have multiple IPIs
  • We have moved to use ISO 639-3 for our language table. While not all languages are exposed at the moment, this gives us a lot more flexibility going forward.

Many thanks to nikki for going way beyond our expectations for testing (and patience!); to Ian McEwen for his continued work on statistics; and to the MusicBrainz team for making this all happen.

If you have a replicated instance of MusicBrainz, please follow these instructions to get your server running on the new schema:

  1. Take down the web server running MusicBrainz, if you’re running a web server.
  2. Turn off cron jobs if you are automatically updating the database via cron jobs.
  3. Make sure your REPLICATION_TYPE setting is RT_SLAVE
  4. Switch to the new code with git fetch origin followed by git checkout v-2012-05-15-schema-change
  5. Run carton install --deployment. If you have not switched your installation to using carton, please read INSTALL.md on how to do this.
  6. Run carton exec -- ./upgrade.sh from the top of the source directory.
  7. Set DB_SCHEMA_SEQUENCE to 15 in lib/DBDefs.pm
  8. Turn cron jobs back on, if needed.
  9. Restart the MusicBrainz web server, if needed.

If you are running a mbslave mirror, check out the latest code and read the upgrade instructions in the README file.

Bug

  • [MBS-3189] – Remove unused ref_count column and related functions
  • [MBS-4616] – Add work language statistics
  • [MBS-4629] – /cover-art page shows no collections
  • [MBS-4637] – Timeline graph won’t graph anything without an entry in statistics/view.js
  • [MBS-4640] – Clicking cover art opens box with “����” (4 U+FFFD)
  • [MBS-4642] – Thickbox CSS interferes with MB CSS
  • [MBS-4647] – Cover art page allows submitting edit with no cover art when JS is off
  • [MBS-4648] – Changing cover art type from “other” to unset causes Internal Server Error
  • [MBS-4678] – upgrade.sh is not ready for testing
  • [MBS-4679] – Internal server error adding secondary types to a release

Improvement

  • [MBS-1485] – Alias types
  • [MBS-1798] – Lyrics language for works
  • [MBS-1799] – Add ISO 639-3 language codes to the database
  • [MBS-1981] – Add blog feed to the home page
  • [MBS-2240] – Aliases: certain locale can be used only once in the list of aliases
  • [MBS-2532] – Allow more than one IPI per artist
  • [MBS-2851] – Timeline graph events should be in the database
  • [MBS-2885] – Allow more than one ISWC per work
  • [MBS-3646] – Split release group attributes into two types
  • [MBS-3788] – Alias improvements
  • [MBS-4625] – Improve wording of cover art tab when cover art comes from relationships
  • [MBS-4676] – Do not allow people entering deprecated relationships

New Feature

  • [MBS-842] – Allow vinyl style track numbers and sides
  • [MBS-1385] – Support unknown end dates
  • [MBS-3704] – Allow adding sort names to artist aliases
  • [MBS-4337] – Make user profile more social: add (optional) fields avatar, gender, birth year, country

Search server index updating paused

Tomorrow’s release requires us to update the main server and the search servers at the same time. This presents a bit of a chicken-and-the-egg problem: We need to build new indexes even before we’ve migrated the database to our new schema.

To accomplish this, I’ve stopped index updating and created a separate database that will allow me to build some indexes that will be a few hours out of date when we release tomorrow. They will be slightly old, but at least we will have indexes that work.

I expect the indexes to be up to date about 3 hours of the release is complete. If you find the indexed search out of date, please use the direct search in the meantime.

Sorry for any inconveniences this may cause.

Universal Music UK launches the Artist Gateway… using MusicBrainz!

Universal Music in the UK just launched their Artist Gateway, a site to showcase their artists and to provide lots of detail about their artists. Universal is using MusicBrainz IDs and data throughout their system to give their site the proper structure and links to elsewhere on the net. To launch it, Universal put out a slick video, complete with a plug for MusicBrainz:

[vimeo http://www.vimeo.com/41151572 w=450&h=253]

This is a huge step for MusicBrainz — in the past the labels didn’t even acknowledge our existence and now they are using our data.

Thanks James Cowdery & crew over at Universal in London!

Upgrading our data licenses

A potential customer just nudged me about our use of now deprecated CC licenses and a deprecated Public Domain dedication. We should really address these issues and upgrade our data licenses. We are currently using the following licenses:

I propose that we move to:

The move to the CC license version 3 allows us to use the latest and most robust version of the Creative Commons license. If you’d like lots more detail on what this change means, please read the CC version 3 license “brief” explanation.

The CC0 grant improves a number of aspects of our old Public Domain dedication, especially in jurisdictions outside the US. Our current dedication isn’t sufficient to renounce any copyright over the data in some countries. The CC0 grant is an improved version that maximizes the global coverage of our rejection of copyright for our data.

Summary in plain english: We’d like to move to updated, more robust licenses. We are not changing what data is available under which license, nor are we taking away any rights that end users already have.

If you have questions or comments, please post them here. If there are no objections to this change, I will make it effective with our May 15th release.

UPDATE: Fixed an incorrect link that Mike from the Creative Commons pointed out. Thanks Mike!

Next schema change release: May 15

We’ve talked a bit about our upcoming schema change release, but we hadn’t nailed down the exact date of the release. Now that we’re tangibly close, we’ve settled on the May 15th as the actual release date.

As a reminder, here are the tickets that will change our schema and here are all the tickets scheduled for the 15th.

Server update, 2012-04-30

We’ve released another set of bug fixes and improvements for the server. Thanks to Joachim LeBlanc, Johannes Weißl and the rest of the MusicBrainz team for helping on this release!

Bug

  • [MBS-2553] – Change wording for no External Links in the sidebar
  • [MBS-4041] – Nonexistent elections cause Internal Server Error
  • [MBS-4544] – Predefined advanced searches on /search no longer work
  • [MBS-4547] – URL cleanup broken on URL edit page
  • [MBS-4563] – Plugin::Diff is broken
  • [MBS-4564] – robots.txt served as octet-stream on test.mb
  • [MBS-4566] – Advanced Search Syntax Link missing from Search
  • [MBS-4573] – Show a message when a label has no releases
  • [MBS-4597] – Merging can fail if entities have aliases with the same locale
  • [MBS-4609] – Comment element on release group in /ws/2 should be removed

Improvement

  • [MBS-1167] – "Read more" for annotation previews should load the full annotation in place, not take you to a different page
  • [MBS-1881] – Move relationship types/attributes/instruments lists
  • [MBS-2479] – Mark approvals differently from normal Yes votes
  • [MBS-2814] – Reports don’t highlight entities with pending edits
  • [MBS-3684] – Improve sorting of the "Releases with superfluous data tracks" report
  • [MBS-4038] – Let users edit their permissions on test servers
  • [MBS-4063] – Exclude stuff marked Single AND UK from the SeparateDiscs report
  • [MBS-4125] – Sort report for creative commons download relationships
  • [MBS-4476] – Provide indication of non-front cover art
  • [MBS-4493] – ISRCs with multiple recordings: Do something about the DEF05 ISRCs
  • [MBS-4530] – Remove the recordings with CC download relationships report
  • [MBS-4560] – "No votes" failed edit should not fail in silence
  • [MBS-4572] – Open up /artist/ in robots.txt

New Feature

  • [MBS-4551] – Display which git branch is active
  • [MBS-4567] – Use markdown for README and INSTALL
  • [MBS-4576] – Allow for google analytics support
  • [MBS-4590] – Use /doc/About instead of /doc/About_MusicBrainz
  • We’ve also improved cover art support slightly.

Sub-task

  • [MBS-4160] – SoundCloud relationship under the External links section