Category: MusicBrainz

Posts related to the MusicBrainz encyclopedia in general: community, data, client, server, etc.

Deprecating MBIDs

This post is an April Fools joke. Rest assured, we have no intention of changing the MBID system that MusicBrainz currently uses.

But, like all good parody news items, there is an element of truth behind this post. The announcement of the Echo Nest API shutdown is real, and with this change you will no longer be able to use the Echo Nest IDs to look up information. This particularly hurts users of the Million Song Dataset, which maps each track to the Echo Nest ID. The new Spotify API isn’t even providing any compatibility api or ID mapping, leaving users to look up 1 million Spotify IDs in the remaining months that the the old Echo Nest api will remain available.

At MusicBrainz, we understand the importance of a stable identifier system. That’s why, 16 years ago, we picked these unwieldy-looking UUID identifiers – that have since proven to have stood the test of time, with room to continue growing. You can look up an MBID made 16 years ago, today – and it will still work another 16 years in the future.

Hello all,

Following Echo Nest’s bold announcement that Echo Nest ids are being replaced by Spotify IDs, we figured it was time to make our own ID change public as well – MBIDs were a fantastic idea 16 years ago, but let’s face it, they’re not the most beautiful thing around, so our MBIDs will now also be replaced by Spotify IDs to help with a proper mapping across tools. Anything without a Spotify mapping will simply get purged. This should greatly simplify the data we have and remove any doubt for some releases whether they exist or not – if they’re on Spotify, they clearly exist!

We would like to commend Echo Nest on their brave leadership in this, giving us the courage to move on from our ancient heritage and try new things. With the speed technologies evolve in this digital age, it can be hard to keep up with things and keep things fresh, but Echo Nest is showing the way forward, and we’re delighted to be able to follow so quickly in their path.

I hope you all will welcome this bold move by our team. We hope to have it ready by next schema change. We know we’re excited! 😀

PS. No, we will not provide a mapping between MBIDs and the new Spotify IDs. We trust our data users to be capable to set things up on their own. Happy hacking! 🙂

Upgrading Postgres for MusicBrainz Live Data Feed users

We’re slowly approaching that time of year: Schema change release time. After skipping our fall update to focus on some internal tasks, we’re ready to have another schema change release in the spring: May 16, 2016

We have started the process to collect features we wish to release for this schema change release and we’ll be publishing that list in the coming weeks. However, we’re contemplating the impact of one more change we’d like to make: Upgrading to a more recent version of Postgres.

Internally we are going upgrade to Postgres 9.5, which was recently released, so we expect that the Postgres team will have worked out the most significant kinks before we’re ready to move to it. However, even though we are moving to 9.5, we are considering the impact on our downstream users/customers who need to make the same or similar change.

While we are moving to version 9.5 of Postgres, we have the option of only adopting features from Postgres 9.4, which means that our downstream users may continue to use Postgres 9.4. However, Postgres 9.5 has some nice features we’d like to use (e.g. UPSERT), so we’re pondering if it is possible for us to require Postgres 9.5 from all of ours Live Data Feed users starting on May 16, 2016. 

We have already informally queried a few of ours users and so far it seems that requiring Postgres 9.5 is feasible. If you are a Live Data Feed user and feel that this requirement of Postgres 9.5 is too much for your and your organization by May 16, 2016, please leave a comment to this blog post!

Schema change release, 2015-05-18 (including upgrade instructions)

Our previously mentioned schema change release is finished! Below will be upgrade instructions, including configuration updates for replication access tokens.

This release does not include UI for several of the schema change patches, which will (hopefully) happen for next release on June 1. The incomplete patches are MBS-7489 (credits for artists in relationships), MBS-4145 (tag upvote/downvote), and MBS-8004 (collections for additional entity types). These patches have had their schema change components finished, but the UI was incomplete or needed more work.

Schema Change Upgrade Instructions

These are largely as previous upgrade instructions, using the tag v-2015-05-18-schema-change. The primary difference is the inclusion of configuring an access token for replication.

  1. Make sure your REPLICATION_TYPE setting is RT_SLAVE and your DB_SCHEMA_SEQUENCE is set to 21 in lib/DBDefs.pm. If you’re running a standalone server, you can run the upgrade, but it may be easier to just import a new data dump!
  2. Ensure you’ve replicated up to the most recent replication packet available with the old schema. (if you’re not sure, run ./admin/replication/LoadReplicationChanges and see what it tells you; if you’re ready to update, it should say “Mismatched schema sequence, 21 (database) vs 22 (replication packet)”).
  3. Take down the web server running MusicBrainz, if you’re running a web server.
  4. Turn off cron jobs if you are automatically updating the database via cron jobs.
  5. Switch to the new code with git fetch origin followed by git checkout v-2015-05-18-schema-change
  6. Run ./upgrade.sh (or carton exec -Ilib -- ./upgrade.sh if you’re using carton, with very old setups).
  7. Run cpanm --installdeps --notest . to ensure your perl-based dependencies are up to date. This release adds a dependency on LWP::Protocol::https, for fetching replication packets from the new server; many systems may already have this installed, but it should be verified.
  8. Set DB_SCHEMA_SEQUENCE to 22 in lib/DBDefs.pm as instructed by the output of ./upgrade.sh
  9. Assuming you have been updating your server with replication, it will now be necessary to configure an access token:
    1. Go to https://metabrainz.org/supporters/account-type and choose your account type as applicable. If you’re an individual, non-commercial user of the data, choose “non-commercial”; if not, choose an applicable tier in the “commercial” section. If you’re not sure of the appropriate tier, make your best guess; it can be adjusted if necessary.
    2. Then, from https://metabrainz.org/profile, create an access token, which should be a 40-character random alphanumeric string provided by the site.
    3. Finally, add this token to lib/DBDefs.pm under the REPLICATION_ACCESS_TOKEN configuration option. The final configuration section should look something like sub REPLICATION_ACCESS_TOKEN { "ck3UpgwgOXhWC6SpFcd99rZOTjzfrei3gQlgZZ9z" }.
    4. Don’t reveal your access token! If you do, inadvertently, you can use the MetaBrainz site to generate a new token, invalidating the old one. (The one in the example above is one I created for myself and then invalidated — don’t get any ideas, it won’t work!)
  10. Turn cron jobs back on, if applicable.
  11. Restart the MusicBrainz web server, if applicable. It’s also recommended you restart memcached.

Finally, the list of bugs closed this release:

Bug

  • [MBS-4436] – Medium titles cannot be longer than 255 charaters

Improvement

  • [MBS-1347] – Implement aliases for release groups, releases and recordings
  • [MBS-7906] – maybe don’t show “”≠null diff. in edit pages
  • [MBS-8279] – Remove empty_artists etc. database functions

New Feature

  • [MBS-8302] – Add Live Data Feed access token support

Task

  • [MBS-8266] – Make medium titles VARCHAR NOT NULL
  • [MBS-8278] – Update DB_SCHEMA_SEQUENCE in DbDefs.pm.sample
  • [MBS-8283] – Remove DB constraint that disallows empty event names

Not included in this list but also relevant is MBS-8349, which while fixed for a previous release, in this release is also applied to old slave servers, which may help performance for some queries.

Picard 1.3.2 released

Picard 1.3.2 is now available on the Picard download page. This again is only a small maintenance release which mainly updates the OS X build to fix crashes some users experienced with Picard 1.3.1. Here is the complete list of fixes:

Bug

  • [PICARD-681] Fixed tags from filename dialog not opening on new installations
  • [PICARD-682] Picard 1.3.1 crashes on OSX on start

For Ubuntu users we have also finally updated our stable PPA.

Picard 1.3.1 released

This is a maintenance release, mostly bug fixes. All users are encouraged to upgrade.

Binary packages are available at http://picard.musicbrainz.org/downloads/

Bug

  • [PICARD-273] – Picard should use the correct Accept header when talking to web services.
  • [PICARD-589] – Picard refuses to load files if any path component happens to be hidden
  • [PICARD-642] – ConfigUpgradeError: Error during config upgrade from version 0.0.0dev0 to 1.0.0final0
  • [PICARD-649] – Windows installer sets working directory to %PROGRAMFILES%\MusicBrainz Picard\locale
  • [PICARD-655] – Last.fm plus tooltip help elements are all messed up
  • [PICARD-661] – Regression: Tagger script for cover art filename does not work anymore
  • [PICARD-662] – Retrieving collections causes AttributeError: release_list
  • [PICARD-663] – Artist name makes it impossible to save

Improvement

  • [PICARD-658] – Support the new pregap and data tracks
  • [PICARD-659] – Set the originalyear tag when loading a release
  • [PICARD-665] – Web service calls to ports 80 and 443 do not need explicit port specification. 443 should be automatically made https.

Schema change upgrade instructions, schema 21

This upgrade shouldn’t be substantially different than past upgrades, now that we’ve fixed a few bugs with the process. To upgrade:

  1. Make sure your REPLICATION_TYPE setting is RT_SLAVE and your DB_SCHEMA_SEQUENCE is set to 20 in lib/DBDefs.pm.
  2. Ensure you’ve replicated up to the most recent replication packet available with the old schema. (if you’re not sure, run ./admin/replication/LoadReplicationChanges and see what it tells you).
  3. Take down the web server running MusicBrainz, if you’re running a web server.
  4. Turn off cron jobs if you are automatically updating the database via cron jobs.
  5. Switch to the new code with git fetch origin followed by git checkout schema-change-20-to-21
  6. Run ./upgrade.sh (or carton exec -Ilib -- ./upgrade.sh if you’re using carton, with very old setups).
  7. Set DB_SCHEMA_SEQUENCE to 21 in lib/DBDefs.pm
  8. Turn cron jobs back on, if needed.
  9. Restart the MusicBrainz web server, if applicable. It’s also recommended you restart memcached.

That’s it! The only real difference from the past is the specific tag to be used: schema-change-20-to-21, which is a couple of fix-up commits past the regular release tag.

Picard 1.3 released

Version 1.3 of Picard has been released today, with some notable features and bug fixes.

This release has few visible changes, but overall performance and stability were much improved. A lot of minor annoying issues were fixed. Cover art code was reworked, and notably one can now enable fallback on release group cover art if no cover art exists for a specific release.

Logging was improved, user can now enable debug mode from About →View Error/Debug Log, it may help to see what is going on if needed, passwords and sensible information were hidden from the log, so user can now safely post his log to get help.

To report any issue concerning this release, please use our bug tracker. As usual you can also get help on forum or on IRC (freenode, #musicbrainz).

Special thanks to Sophist, Wieland Hoffmann, Michael Wiencek, Johannes Dewender, Lukáš Lalinský, Frederik “Freso” S. Olesen, and the whole MusicBrainz Team.
Many thanks to all users, developers, bug reporters and translators. Code contributions since 1.2 are visible on Github Contributors graph.

Unfortunately the OAuth support patch (PICARD-305) didn’t make it in this release.

Picard 1.3 can be downloaded on the Picard’s website.

Be aware that downgrading from 1.3 to 1.2 may lead to configuration compatibility issues, better save your configuration before installing 1.3 if you intent to go back to 1.2.

Among changes since 1.2:

  • The “About” window now displays the versions of libraries used by Picard
  • Picard now correctly handles matching of MP3 files saved in ID3v2.3 tags (which is the version that Microsoft Windows and iTunes both use).
    Note: You may need to re-save your tags once to get them to match in future.
  • A sort tags plugin is now provided as tag data is no longer displayed sorted by default.
  • A new tag, musicbrainz_releasetrackid, containing the MusicBrainz Track MBID introduced in the May 2013 schema change release, is now written to files.
  • Add %_recordingtitle% (PICARD-515)
  • Fix plugin install bugs (PICARD-444)
  • Fix Options → File naming examples to handle primary/secondary release types (PICARD-516)
  • A new advanced option is available to permanently set the starting directory for the file browser and “Add files/folder” buttons.
  • Requests to Musicbrainz against your own account e.g. for collections are now handled through SSL (PICARD-337)
  • Refresh of Albums using Ctrl-R and selection of Other Releases are now more responsive during batch lookups.
  • Main window is now emitting a “selection_updated” signal, plugin api version bumps to 1.3.0
  • Append system information to user-agent string
  • Compilation tag/variable functionality (for tagging & file naming) has been split into two:
    – %compilation% is now aligned with iTunes, and set only for Various Artists type compilations
    – %_multiartist% variable now indicates whether this release has tracks by multiple artists
    (in order to prepend the artist name to the filename as shown in the default file naming script)
  • autodetect the CD drive on Mac OS X (PICARD-123)
  • Ignore directories and files while indexing when show_hidden_files option is set to False (PICARD-528)
  • Add ignore_regex option which allows one to ignore matching paths, can be set in Options → Advanced (PICARD-528)
  • Added an “artists” multi-value tag to track metadata, based on the one in Jaikoz, which contains the individual artist names from the artist credit. Also useful in scripts (joining phrases like ‘feat:’ are omitted) and plugins.
  • Added “_artists_sort“, “_albumartists“, “_albumartists_sort” variables for scripts and plugins.
  • Made Picard use the country names also used on the MusicBrainz website (PICARD-205)
  • New setup.py command `get_po_files` (Retrieve po files from transifex)
  • New setup.py command `regen_pot_file` (Regenerate po/picard.pot)
  • New Work tag (which for Classical music is often different from the track title) saved as ID3 TOAL tag.
  • New Composer Sort Order tag (variable %composersort%).
  • Improve the Other Releases list to prioritise and separate releases which match the correct number of tracks and your Options → Metadata → Prefered Releases settings for Country and Format.
  • New %_absolutetracknumber% variable numbering tracks sequentially regardless of disc structure (so you can numbers tracks on multi-disc releases without a disc number)
  • Support dropping image directly from Google image results to cover art box
  • Add %_musicbrainz_tracknumber% to hold track # as shown on MusicBrainz release web-page e.g. vinyl/cassette style A1, A2, B1, B2
  • Show the ID3 version of the file in the Info… dialog (Ctrl-I) (PICARD-218)
  • Fixed a bug where Picard crashed if a MP3 file had malformed TRCK or TPOS tags (PICARD-112)
  • Add –files option to setup.py build_ui, used to force .ui to .py regeneration (PICARD-566)
  • New setup.py command `update_constants` (Regenerate countries.py and attributes.py)
  • Made Picard use release groups, medium formats and cover art types also used on the MusicBrainz website
  • Use MusicBrainz Server translations for release groups, medium formats and cover art types
  • Add checkbox to toggle debug at runtime in log/debug view dialog
  • Add a plugin to add Artist Official Homepage relationships to the website tag (ID3 WOAR tag)
  • Add integrated functions $eq_any, $ne_all, $eq_all, $ne_any, $swapprefix and $delprefix.
  • Add %_performance_attributes%, containing performance attributes for the work e.g. live, cover, medley etc.
  • Use $inmulti in file naming scripts i.e. …$if($inmulti(%_performance_attributes%,medley), (Medley),)
  • Add optional `priority` parameter to `register_album_metadata_processor()` and `register_track_metadata_processor()`
    Default priority is `PluginPriority.NORMAL`, plugins registered with `PluginPriority.HIGH` will be run first, plugins registered with `PluginPriority.LOW` will run last
  • Add Standardise Performers plugin to convert e.g. Performer [piano and guitar] into
    Performer [piano] and Performer [guitar].
  • Add support for release group cover art fallback (PICARD-418, PICARD-53)
  • Add a clear button to search box
  • Honour preferred release formats when matching AcoustIds (PICARD-631)
  • Prevent ZeroDivisionError in some rare cases (PICARD-630)

About next stable Picard release

Picard team decided to release a new stable version of Picard before the end of the summer.
To help us, advanced users, translators and developers are encouraged to:

Exact release date is not yet known, and features aren’t yet frozen, but the only important change we’ll try to get in before final release is OAuth support (PICARD-305), we still accept minor patches until feature freeze is announced.

Since Picard is now using some translation resources from MusicBrainz project, translators should make sure following resources are up-to-date:

A simplified list of changes made since 1.2 can be read there.

Be aware that downgrading from 1.3 to 1.2 may lead to configuration compatibility issues, better save your configuration before using 1.3 branch if you intent to go back to 1.2.

 

 

Fire damages the Internet Archive

A fire at the Internet Archive (our friends!) has caused $600,000 in damage. Fortunately no one was harmed and no data was lost:

A fire at the Internet Archive’s San Francisco scanning center has destroyed an estimated $600,000 of digitization and scanning equipment. Fortunately no one was injured in the blaze, but the property damage has ruined “some physical materials” that were yet to be digitized, and restricted the nonprofit organization’s ability to record the history of the web.

MusicBrainz just donated $50 to the Internet Archive and asks you to consider making a donation as well.