Category: Development

Changes for upcoming schema change release: 2013-10-14

For our fall schema change release we’re going to fix the issues listed in this Schema change, 2013-10-14 fix version. This schema change, much like our previous fall schema changes, is going to be a little lighter than our spring changes. The two big changes that you should be aware of are:

  • MBS-6046: Remove PUID support. PUIDs have been deprecated for quite some time in favor of using AcoustID. This change completes our switchover to AcoustID.
  • MBS-6068: Remove _name tables. With NGS we introduced tables that isolated all of the name strings for a given table, in order to reduce duplicate strings in our database. Sadly, this proved to be more pain that gain, so we’re going to remove them. This will make writing queries for our schema much easier. Sadly, this means that if you use the MusicBrainz database directly, (as opposed to using our Web Service, which we recommend) then you will likely need to update your SQL queries.

The rest of the changes will have a much smaller impact: Adding a disambiguation column for areas, adding places, and adding the ability to mark a relationship as deprecated. We’re also making some minor changes to our non-replicated tables that will not be visible to our Live Data Feed users, but are more convenient to make during a schema change.

The scheduled date for this release is Monday, 14th October, 2013.

Our RDFa dilemma

A few years ago Queen Mary University was awarded a grant to implement modern RDF support in MusicBrainz. The RDFa portion was implemented on our server and has been in our pages for quite some time.

However, the code to implement RDFa is brittle and has not been maintained through a number of schema changes and is quite broken at this point in time. When wondering if we should fix this or remove it, we could find no one or no application that we know of, that makes use of the embedded RDFa in our pages. And no one stepped up to fix it and the author of this code is not responding to emails inquiring about this.

At this point, we’re ready to remove the broken code from our pages in an effort to remove technical debt that has accumulated over the past few years. If you care about RDFa support in our pages, please speak up now. Ideally anyone speaking up would also volunteer to adopt the RDFa code and see it through life as our schema changes.

We’re going to take the HTTPS plunge!

Yesterday in our dev meeting we agreed to take the HTTPS plunge for all of our web site traffic in as little as 2 weeks time. This means that all web site traffic (not the web service) will be served over HTTPS; if you visit any MusicBrainz HTTP URL (e.g. http://musicbrainz.org ) you will be redirected to the equivalent HTTPS URL (e.g. https://musicbrainz.org ). This will not be applied to our web services, you’ll still be able to access those with HTTP. However, we do encourage all of our web service users to make use of HTTPS when possible.

We have one bug to address before we make this switch. And if we can find a sufficient fix for this in time, we’re going to make the HTTPS switch on 16 September 2013. If we can’t find an acceptable fix, we’ll have to postpone this switchover.

If for some reason you can see that switching all web site traffic to HTTPS is a bad idea, please leave us a comment ASAP.

Server Update 2013-09-02

We have a tiny release to kick off September, as we’ve been focusing quite heavily on tightening up the reliability and ease-of-maintenance of the servers themselves. However, we still have a few important bug fixes to release. Here’s what’s changed:

Bug

  • [MBS-4730] – Sort names for artist search hint aliases are displayed, but cannot be edited.
  • [MBS-6218] – When a Person with a gender is merged into a Group, ModBot ISEs.
  • [MBS-6561] – End column empty in search results for ended artists with no end date
  • [MBS-6635] – Several reports not updated since May 15
  • [MBS-6690] – JSON web service ISE with inc=aliases
  • [MBS-6698] – Wiki excerpts truncated by mid-sentence initials (eg, of names) that are mistaken for periods ending a sentence.

The Git tag for this release is v-2013-09-02

Server Update, 2013-08-19

Another two weeks, another release! A variety of bug fixes, primarily, and some assorted improvements. For those of you using replication on resource-constrained servers, this release includes code to make ProcessReplicationChanges run in constant memory, which should hopefully be an improvement.

With help from Michael Wiencek and the rest of the MusicBrainz team, here’s what we’ve done in the last fortnight:

Bug

  • [MBS-1549] – Guess Case: “¿” in sentence mode
  • [MBS-2151] – Editing Track Times should remove existing value
  • [MBS-2959] – Release-group XML first release date is using incorrect sorting logic for incomplete dates
  • [MBS-4389] – Editing artist credits shows “undefined” as disambiguation comment
  • [MBS-4788] – Editing → Add Release allows user to set release type even when existing release group has been selected
  • [MBS-5508] – Uploading cover art from https pages causes a warning about insecure pages
  • [MBS-5965] – Remove relationship edits store translated attributes
  • [MBS-6192] – Internal server error on ws query if offset is non-numeric
  • [MBS-6220] – tooltip (title) shows “&” instead of “&”
  • [MBS-6243] – “In x hours” is only shown for edit expiry times for the same day
  • [MBS-6418] – Relationship editor allows adding more relationships than the server can handle
  • [MBS-6546] – When the search server reports an internal server error the client receives a 400 status code
  • [MBS-6558] – Internal server error logging in with a Unicode username
  • [MBS-6570] – Removing relationship dates in the relationship editor fails
  • [MBS-6571] – Filename display in cover art edits is hardcoded to jpg
  • [MBS-6588] – “Direct database search” returns the same work as 2 different names
  • [MBS-6591] – JSON: Composer and release lookup
  • [MBS-6610] – Various Artist Alias and Tags should not be shown unless looking up Various Artist artist endpoint
  • [MBS-6622] – All log in pages are not served via HTTPS
  • [MBS-6628] – ModBot is unable to close some merge label edits
  • [MBS-6630] – Remove button when uploading covers is shown after the image has been submitted
  • [MBS-6632] – Relationship documentation examples don’t work with release groups
  • [MBS-6637] – http://musicbrainz.org/release/add doesn’t redirect to beta when desired
  • [MBS-6638] – ‘use beta site’ on main site will unset preference
  • [MBS-6650] – Editing a relationship type allows submitting the form with no changes
  • [MBS-6656] – Relationship editor fails when counting number of selected recordings
  • [MBS-6657] – Internal server error when using a negative limit
  • [MBS-6667] – Beta: No edit note field in the relationship editor
  • [MBS-6677] – Donations from random end users dropped to nearly nil.
  • [MBS-6679] – beta: ISE displaying artist edits

Improvement

  • [MBS-193] – Add open edit and cancelled edit stats to users
  • [MBS-1820] – Release search results should include more information
  • [MBS-2108] – More detail for works quicksearch
  • [MBS-3204] – Better name/explanation or warning for type “Pseudo-Release”
  • [MBS-6644] – ProcessReplicationChanges should be able to run in low memory environments

Search server release: 2013-08-14

Paul Taylor worked on the latest release of our search server — thanks for fixing these bugs, Paul! The following issues were addressed:

Release Notes – MusicBrainz Search Server – Version 2013-08-14

Bug

  • [SEARCH-306] – Advanced search can’t find areas by alias
  • [SEARCH-315] – REGRESSION:Recording index is not outputting Release artist credit when different to recoridng artist credit (i.e. Various Artist)

Server Update, 2013-08-05

It’s a new month, which means it’s time for a a new release of MusicBrainz! This week we’re releasing a new version of the Cover Art Archive uploader, which now lets you upload multiple files at once, and also supports more file formats (including PNG and GIF):

Uploading multiple files to the Cover Art Archive

Along with this, we’ve also solved a good selection of bugs, and have added a few little improvements along the way too. With help from Nicolás Tamargo and the rest of the MusicBrainz team, here’s what we’ve done in the last fortnight:

Bug

  • [MBS-4669] – Webservice returns an ASIN that was added in a voted down edit
  • [MBS-4981] – Cover art in sidebar links to CAA /front when no front is available
  • [MBS-5625] – Date display problem in edit listings
  • [MBS-6148] – Adding a disc ID automatically fills in durations without creating an edit
  • [MBS-6262] – Release events section of the release editor is all over the place
  • [MBS-6395] – medium_index not updating on medium inserts
  • [MBS-6442] – robots.txt allows /login
  • [MBS-6446] – Internal server error trying to register the username “0”
  • [MBS-6469] – Selecting an example for a relationship type does not enable the “add example” button
  • [MBS-6490] – 404 error when accessing /browse
  • [MBS-6494] – Adding area of death should mark ended
  • [MBS-6496] – Add release > Release duplicates (1CD) as a starting point > Add medium (CD2) : this CD2 gets CD1 recordings !
  • [MBS-6549] – Release Editor Seeding : events.n.country = “JP” doesn’t work
  • [MBS-6563] – Text “Cover art from Amazon” not shown under cover art, when other (non‐front) cover art have been uploaded to CAA
  • [MBS-6565] – mm.musicbrainz.org should redirect to musicbrainz.org
  • [MBS-6566] – Missing elements in webservice lookup
  • [MBS-6586] – HASH(0xe228218) displaying a move disc ID edit
  • [MBS-6596] – Release country showing as [removed] in edits
  • [MBS-6600] – Internal server error displaying track 404 page
  • [MBS-6604] – Unable to set up ANY recording-to-recording relationship in Relationship Editor
  • [MBS-6605] – HASH(hex_value) is being displayed on the ISRC page instead of ISRC value
  • [MBS-6606] – Edit relationship edits always apply even if the underlying relationship has changed
  • [MBS-6608] – Internal server error viewing split artist page when logged out
  • [MBS-6611] – Cover art uploader shows wrong/confusing filesizes
  • [MBS-6614] – No way to remove unrecognised file in the cover art uploader
  • [MBS-6615] – Cover art uploader allows selecting more images after starting upload
  • [MBS-6616] – Comment field is not hidden while/after uploading an image
  • [MBS-6618] – Cover art uploader ignores auto-edit checkbox
  • [MBS-6619] – Cover art uploader doesn’t seem to handle SlowDown errors
  • [MBS-6620] – Don’t accept unsupported images or files which aren’t images
  • [MBS-6623] – No way to resubmit form when cover art fails to upload
  • [MBS-6627] – Cover art uploader doesn’t correctly upload GIFs/PNGs

Improvement

  • [MBS-3618] – Force a check when adding a huge number of tracks
  • [MBS-3823] – Track Parser’s “Done” Button should not take any action other than hiding the track parser window
  • [MBS-4375] – Cover art archive: Upload multiple files at a time
  • [MBS-4376] – Image upload should use AJAX instead of an iframe.
  • [MBS-4442] – Be able to adjust position of new image without constantly having to move the mouse
  • [MBS-4645] – Improve error handling when uploads to the IA fail
  • [MBS-5587] – Release-group front cover art support : URL of image source is caa.org/release not caa.org/release-group/
  • [MBS-6256] – “Example relationships” for relationship documentation should use entity autocompletion
  • [MBS-6408] – Add a view for release events
  • [MBS-6612] – Show previews in the cover art uploader
  • [MBS-6613] – Improve display of “cancel” button in the cover art uploader

New Feature

  • [MBS-4849] – Drag & drop based file uploader for album artwork

Task

  • [MBS-6584] – Remove cruft from Controller::$Entity

The Git tag for this release is v-2013-08-05.

libdiscid 0.5.2 released

This is a bugfix release of libdiscid.

There are already lots of changes (features) ready for 0.6.0, but I did want to release these fixes separately, since they have a high impact on the platforms involved.
0.6.0 is probably ready in a couple of weeks.

Bug

  • [LIB-50] – Solaris: segfault in mb_disc_load_toc
  • [LIB-51] – Mac: Unable to read ISRC for track

New Feature

  • [LIB-26] – disc access test suite
  • print time information in the discid example (zas)

Information, documentation and other links are at:
http://musicbrainz.org/doc/libdiscid
That includes builds for Windows and Mac OS X.

New virtual machine available via BitTorrent

We have finally created a new version of our Virtual Machine and we’re currently seeding it via BitTorrent. If you have some spare bandwidth and would like to help seed this torrent, please join it and leave your client running.

You can find the torrent file here. (edit: for the curious/disk-limited, the downloaded .ova is 10.13GB). Please read the instructions for how to use this VM.

You should consider this VM as beta quality as it not been tested at all. Let’s hope for the best!

UPDATE: We’ve got a workaround for getting this VM to work in VirtualBox. It works just fine under VMWare Fusion and Player.

UPDATE 2: We’ve added a second tracker to the BitTorrent.

Server update, 2013-07-22

We’ve just finished deploying the second update to musicbrainz.org for July. Continuing the trend, this release is primarily bug fixes with a few new improvements. Notably, people should be happy to hear that the Cover Art Archive now supports PNGs and GIFs, ISRC editing has become easier, and the release and relationship editors should be a little bit more stable. Many thanks to Nicolás Tamargo along with the MusicBrainz team for their continued work on MusicBrainz. Here’s what’s changed:

Bug

  • [MBS-3132] – Problems with track artist credits when seeding the release editor
  • [MBS-5310] – Artist aliases should not be shown for work relations when doing a release lookup
  • [MBS-5409] – Internal server error when artist begin year is 0
  • [MBS-5637] – Regression: Recording merges can result in duplicate relationships
  • [MBS-6058] – Artist Subscriptions overview page is contradicting itself
  • [MBS-6194] – regression : drums release relationships were invisible as long as there was at least one recording drums relationship
  • [MBS-6235] – Internal server error when using date of birth without century in profile
  • [MBS-6273] – Country names are no longer shown in the sidebar or in edits
  • [MBS-6275] – Diffs no longer work for release dates
  • [MBS-6276] – Release dates are badly aligned in edits
  • [MBS-6280] – Registering an account ignores DB_READ_ONLY
  • [MBS-6282] – Uploading cover art ignores DB_READ_ONLY
  • [MBS-6283] – Voting on edits ignores DB_READ_ONLY
  • [MBS-6284] – Cancelling edits ignores DB_READ_ONLY
  • [MBS-6285] – Verifying an email address ignores DB_READ_ONLY
  • [MBS-6286] – Editing profile/preferences/password ignores DB_READ_ONLY
  • [MBS-6288] – Adding ISRCs ignores DB_READ_ONLY
  • [MBS-6289] – Editing CD stubs ignores DB_READ_ONLY
  • [MBS-6290] – Removing subscriptions ignores DB_READ_ONLY
  • [MBS-6354] – Possible to edit after unsetting email
  • [MBS-6360] – Internal server error when trying to authenticate via digest auth with non-ASCII username
  • [MBS-6369] – No link to area when editing an artist, a label or anything that has an area inline search
  • [MBS-6401] – Regression: Default release group cover art is not (always?) the original release
  • [MBS-6403] – Resetting password ignores DB_READ_ONLY
  • [MBS-6410] – Internal Server Error
  • [MBS-6413] – Areas’ releases tab should show "This area is not currently associated with any releases" if empty
  • [MBS-6417] – JSON API does not return disc number information for media
  • [MBS-6426] – ModBot unable to close old edit release edits that cleared both date and country
  • [MBS-6454] – Pressing enter when editing a relationship type does not submit the form
  • [MBS-6455] – Internal server error displaying edit relationship type edits
  • [MBS-6468] – Relationship type documentation pages are not linked in appropriate places
  • [MBS-6486] – ModBot no longer enters remove relationship edits when using "Split into separate artists"
  • [MBS-6498] – Internal server error displaying an historic set track lengths edit
  • [MBS-6531] – area/not_found.tt doesn’t exist
  • [MBS-6536] – OAuth token expiration ISE
  • [MBS-6569] – JSON file uploaded to the IA has the wrong file extensions for GIFs/PNGs
  • [MBS-6578] – ModBot is unable to close Edit::Area::Merge edits

Improvement

  • [MBS-2411] – Login page should be encrypted (SSL/TLS)
  • [MBS-4114] – Cover art archive: Support .gif
  • [MBS-4671] – Enter "Remove relationship" edits as the same editor instead of ModBot when splitting artists
  • [MBS-5868] – Move ISRC editing to the edit recording page
  • [MBS-6022] – Cover art archive: Support .png UI Changes
  • [MBS-6414] – Wikidata URL cleanup should also remove the "fragment" part of the URL
  • [MBS-6457] – Relationship editor deals confusingly with deprecated relationships
  • [MBS-6575] – Provide more debugging information if the release editor fails to create edits

The Git tag for this release is v-2013-07-22.