Month: April 2013

Server Update 2013-04-22 and a Notice Regarding Passwords

We’ve just finished deploying a new update to the MusicBrainz website, but before covering release notes we’d like to clarify a few points regarding the recent password leak.

There have been a few reports suggesting that MusicBrainz was attacked or compromised, but we can assure that this is not the case. As stated in our original blog post, we accidentally included private information in our otherwise routine public backups. As the majority of MusicBrainz data is available under the Creative Commons BY-NC-SA or CC0, these backups are intended to be public – the mistake was the human error in accidentally including password hashes.

A few of you also mentioned that we didn’t respond in a timely fashion in alerting users via emails. We completely agree with these comments, and apologise for not being able to respond quicker. Unfortunately, we did not have the infrastructure to do such mass mailing, and as we didn’t want to rush this out to the point of making yet more errors, it took us a little longer than we all would have liked.

With that out of the way, here’s what we’ve done in the last fortnight. Many thanks to Michael Wiencek, Nicolás Tamargo, and the rest of the MusicBrainz team for their work on this release.

Bug

  • [MBS-4277] – Guess Case: Keep uppercase option should be clearer
  • [MBS-4468] – “Enter vote” button incorrectly also adds edit note
  • [MBS-5597] – Funky Caps inConsisTency in Edit search paGes (presets or not)
  • [MBS-5832] – Rating not showing on Merge Recordings page
  • [MBS-5967] – No feedback after editing a user
  • [MBS-5998] – Barcodes incorrectly represented in mo RDF
  • [MBS-6077] – Reorder tracks edit not working
  • [MBS-6081] – Viewing MusicBrainz Events and Cover Art information makes the CAA launch event appear to the left of the actual graph
  • [MBS-6082] – “Remove Label” edit page has broken documentation link
  • [MBS-6100] – Internal server error with webservice requests which require authentication
  • [MBS-6111] – Amazon taking precedence over CAA after merging releases
  • [MBS-6121] – Logging in with non-ascii password throws exception
  • [MBS-6122] – Unable to merge release groups when cover art is set
  • [MBS-6169] – Check for edit user uses the wrong sub

Improvement

  • [MBS-4632] – Link to archive.org item from cover art tab
  • [MBS-4658] – Report: duplicate ISWCs
  • [MBS-5261] – Link to How to Add Disc IDs from the Disc ID tab of releases
  • [MBS-5667] – Rename “Other release groups”
  • [MBS-5800] – Remove restriction on entering standalone recordings
  • [MBS-6001] – Make the autoeditor election overview more useful
  • [MBS-6034] – label search is missing country column
  • [MBS-6086] – Setting cover art type (from nothing to something) should be an autoedit
  • [MBS-6109] – Work search doesn’t show type
  • [MBS-6110] – Relate To URL should auto-identify audiojelly.com as “can be purchased for download” relation.
  • [MBS-6119] – Seeing vote counts on closed elections shouldn’t require log in

New Feature

  • [MBS-1891] – Anchors/Permalinks on Edit Notes
  • [MBS-5737] – Notification when RE has created a release

Task

  • [MBS-6096] – Add soundtrackcollector.com to the Other Databases whitelist

The Git tag for this release is v-2013-04-11.

libdiscid 0.5.0 released

A new libdiscid release was made available today.

Changes:

– LIB-29: add read_sparse() for faster reading again
– LIB-35: add HAVE_SPARSE_READ and VERSION_* defines
– LIB-36: hide internal symbols on Linux/Unix
– LIB-34: distmac and distwin32 cmake targets

The important change is the read_sparse() function:

Philipp Wolfer added a read_sparse() function. With this function you can either only read the TOC or specifiy which features of the disc you want to read.
The normal read() also extracts ISRCs starting with 0.3.0.
You might want to change existing applications to use read_sparse if you care about performance and don’t use ISRCs. The TOC is usually cached, so read_sparse() can be faster (0,5 vs. 3 seconds measured).
The difference is only in where the time is spent. It doesn’t really save overall time, but the TOC is read right when the disc is inserted so no additional disc access is performed when using the TOC in your application.
To make it possible to keep using read() when read_sparse() is not available we provide the HAVE_SPARSE_READ define, which can be used like that:

#ifndef DISCID_HAVE_SPARSE_READ
#define discid_read_sparse(disc, dev, i) discid_read(disc, dev)
#endif
…
discid_read_sparse(disc, device, 0)

We also provide defines for the libdiscid version numbers.
However, you should rather test for features/functions in the build files and create specific defines for your use case.
The above define is only provided as a convenience for read_sparse().

There are more details about the other changes in the full announcement mail.

If you didn’t follow the musicbrainz-devel list:
This year brought several new releases for libdiscid, starting with ISRC and MCN support in libdiscid 0.3.0. Applications using libdiscid 0.2.2 (or even lower) still work with libdiscid 0.5.0.

Information, documentation and other links are at:
http://musicbrainz.org/doc/libdiscid
That includes builds for Windows and Mac OS X.

Server Update, 2013-04-08

We’ve just finished pushing out another fortnight of work to the main MusicBrainz servers. While work continues on the forthcoming schema change release, we have fixed a few bugs and added a few improvements. Thanks to Lukáš Lalinský and the rest of the MusicBrainz team for their work on this release! Here’s what’s changed:

Bug

  • [MBS-5524] – Edit relationship type edits are missing information for identifying the relationship type
  • [MBS-5830] – Attributes are not shown in relationship type edits
  • [MBS-5964] – Artist credit’s sort name tooltip doesn’t update after edit
  • [MBS-5980] – Cover art comments can make the cover art page look very unstructured
  • [MBS-5993] – name variation checks happen after HTML entity conversion
  • [MBS-6019] – Some FreeDB data getting wrongly imported from the dumps
  • [MBS-6021] – Track count search in Add Disc uses full release count for multi-disc releases
  • [MBS-6038] – "See all {num} {entity}" is just wrong i18n-wise
  • [MBS-6042] – Release country field in edit search breaks after doing a search
  • [MBS-6043] – In front page "Recent Additions" the text for the cover art pop-up labels (HTML title attribute) is wronlgy escaped
  • [MBS-6075] – An error occured while loading this edit

Improvement

  • [MBS-5613] – Set cover art page does not show enough information
  • [MBS-6039] – Make it possible to display attributes after the link phrase

Sub-task

  • [MBS-4258] – Make dates translatable

The Git tag for this release is v-2013-04-08.

Potential Security Leak

What Happened?

On March 29th 2013 we discovered that one of the MusicBrainz database dumps contained password hashes for a large portion of MusicBrainz accounts. While we don’t believe that these password hashes are either useful or widely distributed, we are requiring all users change their passwords.

What Data Was Leaked?

bcrypt password hashes, with a cost parameter of 8, for all accounts as of March 25th 2013.

Why Did This Happen?

We’ve recently began work on a long standing ticket against MusicBrainz server – MBS-357, “don’t store passwords in clear text”. We’re going to be moving away from clear text passwords, and we’ve decided to use one of the current industry standards for hashing passwords – bcrypt. Using bcrypt means that MusicBrainz will store only the hashes of passwords, which in laymans terms is a “fingerprint” of the password. Hashing means that we never store the actual password, but only the hash. There are many hashing functions available, and bcrypt is designed to be an expensive hash to compute with an adjustable “cost” – this makes it very hard to find out what the original password was via brute force attacks.

While this does mean that it’s hard to extract passwords from the hashes, the initial round of hashing passwords to move away from clear text is time consuming. As such, we built a small program that would gradually hash passwords over the course of a few days in order to make the switch from clear text passwords to secure password hashes done with as little downtime as possible.

This script hashed the password into the bcrypt_password column for all editors, and would also be notified when users changed their password in order to update the hash. Unfortunately, our database dump scripts sanitize this data by excluding data after-the-fact, rather than declaring what data to dump before running the script. As such, it dumped the entire editor table with the new column, as we forgot to add a rule to exclude this column.

Our Response

The database dumps that contain this data were promptly deleted, and have been replaced with correctly sanitized database dumps. Unfortunately logs from this server do show that this database dump was downloaded, and as we have no real indication of where this data now is, we’re treating this seriously. We have adjusted our database dumping scripts to be very specific about exactly which data they should export, so that in the future we will not leak private data by making the same mistake again.

We’re extremely sorry about this mistake, and while we don’t believe this data should allow attackers to retrieve user passwords, we can’t be 100% certain. As such, we require that all users change their password as soon as possible.

Search server fixes released

Last week’s search server release had some bugs that we decided should be fixed sooner than later. Paul Taylor rose to the challenge and fixed 4 important bugs and we just finished releasing the updated code. Thanks for your efforts, Paul!

Release Notes – MusicBrainz Search Server – Version 2013-04-04

Bug

  • [SEARCH-279] – Seach server returning wrong results
  • [SEARCH-280] – Artist search DAVID BOWIE → FRANZ SCHUBERT (score 100) !? Bowie (score 0)
  • [SEARCH-281] – If set explain=true option with dismax search it actually does a non-indexed search

Improvement

  • [SEARCH-267] – Create new rewrite method for Dismax FuzzySearch