Category: MusicBrainz

Posts related to the MusicBrainz encyclopedia in general: community, data, client, server, etc.

Upgrading Postgres for MusicBrainz Live Data Feed users

We’re slowly approaching that time of year: Schema change release time. After skipping our fall update to focus on some internal tasks, we’re ready to have another schema change release in the spring: May 16, 2016

We have started the process to collect features we wish to release for this schema change release and we’ll be publishing that list in the coming weeks. However, we’re contemplating the impact of one more change we’d like to make: Upgrading to a more recent version of Postgres.

Internally we are going upgrade to Postgres 9.5, which was recently released, so we expect that the Postgres team will have worked out the most significant kinks before we’re ready to move to it. However, even though we are moving to 9.5, we are considering the impact on our downstream users/customers who need to make the same or similar change.

While we are moving to version 9.5 of Postgres, we have the option of only adopting features from Postgres 9.4, which means that our downstream users may continue to use Postgres 9.4. However, Postgres 9.5 has some nice features we’d like to use (e.g. UPSERT), so we’re pondering if it is possible for us to require Postgres 9.5 from all of ours Live Data Feed users starting on May 16, 2016. 

We have already informally queried a few of ours users and so far it seems that requiring Postgres 9.5 is feasible. If you are a Live Data Feed user and feel that this requirement of Postgres 9.5 is too much for your and your organization by May 16, 2016, please leave a comment to this blog post!

Schema change release, 2015-05-18 (including upgrade instructions)

Our previously mentioned schema change release is finished! Below will be upgrade instructions, including configuration updates for replication access tokens.

This release does not include UI for several of the schema change patches, which will (hopefully) happen for next release on June 1. The incomplete patches are MBS-7489 (credits for artists in relationships), MBS-4145 (tag upvote/downvote), and MBS-8004 (collections for additional entity types). These patches have had their schema change components finished, but the UI was incomplete or needed more work.

Schema Change Upgrade Instructions

These are largely as previous upgrade instructions, using the tag v-2015-05-18-schema-change. The primary difference is the inclusion of configuring an access token for replication.

  1. Make sure your REPLICATION_TYPE setting is RT_SLAVE and your DB_SCHEMA_SEQUENCE is set to 21 in lib/DBDefs.pm. If you’re running a standalone server, you can run the upgrade, but it may be easier to just import a new data dump!
  2. Ensure you’ve replicated up to the most recent replication packet available with the old schema. (if you’re not sure, run ./admin/replication/LoadReplicationChanges and see what it tells you; if you’re ready to update, it should say “Mismatched schema sequence, 21 (database) vs 22 (replication packet)”).
  3. Take down the web server running MusicBrainz, if you’re running a web server.
  4. Turn off cron jobs if you are automatically updating the database via cron jobs.
  5. Switch to the new code with git fetch origin followed by git checkout v-2015-05-18-schema-change
  6. Run ./upgrade.sh (or carton exec -Ilib -- ./upgrade.sh if you’re using carton, with very old setups).
  7. Run cpanm --installdeps --notest . to ensure your perl-based dependencies are up to date. This release adds a dependency on LWP::Protocol::https, for fetching replication packets from the new server; many systems may already have this installed, but it should be verified.
  8. Set DB_SCHEMA_SEQUENCE to 22 in lib/DBDefs.pm as instructed by the output of ./upgrade.sh
  9. Assuming you have been updating your server with replication, it will now be necessary to configure an access token:
    1. Go to https://metabrainz.org/supporters/account-type and choose your account type as applicable. If you’re an individual, non-commercial user of the data, choose “non-commercial”; if not, choose an applicable tier in the “commercial” section. If you’re not sure of the appropriate tier, make your best guess; it can be adjusted if necessary.
    2. Then, from https://metabrainz.org/profile, create an access token, which should be a 40-character random alphanumeric string provided by the site.
    3. Finally, add this token to lib/DBDefs.pm under the REPLICATION_ACCESS_TOKEN configuration option. The final configuration section should look something like sub REPLICATION_ACCESS_TOKEN { "ck3UpgwgOXhWC6SpFcd99rZOTjzfrei3gQlgZZ9z" }.
    4. Don’t reveal your access token! If you do, inadvertently, you can use the MetaBrainz site to generate a new token, invalidating the old one. (The one in the example above is one I created for myself and then invalidated — don’t get any ideas, it won’t work!)
  10. Turn cron jobs back on, if applicable.
  11. Restart the MusicBrainz web server, if applicable. It’s also recommended you restart memcached.

Finally, the list of bugs closed this release:

Bug

  • [MBS-4436] – Medium titles cannot be longer than 255 charaters

Improvement

  • [MBS-1347] – Implement aliases for release groups, releases and recordings
  • [MBS-7906] – maybe don’t show “”≠null diff. in edit pages
  • [MBS-8279] – Remove empty_artists etc. database functions

New Feature

  • [MBS-8302] – Add Live Data Feed access token support

Task

  • [MBS-8266] – Make medium titles VARCHAR NOT NULL
  • [MBS-8278] – Update DB_SCHEMA_SEQUENCE in DbDefs.pm.sample
  • [MBS-8283] – Remove DB constraint that disallows empty event names

Not included in this list but also relevant is MBS-8349, which while fixed for a previous release, in this release is also applied to old slave servers, which may help performance for some queries.

Picard 1.3.2 released

Picard 1.3.2 is now available on the Picard download page. This again is only a small maintenance release which mainly updates the OS X build to fix crashes some users experienced with Picard 1.3.1. Here is the complete list of fixes:

Bug

  • [PICARD-681] Fixed tags from filename dialog not opening on new installations
  • [PICARD-682] Picard 1.3.1 crashes on OSX on start

For Ubuntu users we have also finally updated our stable PPA.

Picard 1.3.1 released

This is a maintenance release, mostly bug fixes. All users are encouraged to upgrade.

Binary packages are available at http://picard.musicbrainz.org/downloads/

Bug

  • [PICARD-273] – Picard should use the correct Accept header when talking to web services.
  • [PICARD-589] – Picard refuses to load files if any path component happens to be hidden
  • [PICARD-642] – ConfigUpgradeError: Error during config upgrade from version 0.0.0dev0 to 1.0.0final0
  • [PICARD-649] – Windows installer sets working directory to %PROGRAMFILES%\MusicBrainz Picard\locale
  • [PICARD-655] – Last.fm plus tooltip help elements are all messed up
  • [PICARD-661] – Regression: Tagger script for cover art filename does not work anymore
  • [PICARD-662] – Retrieving collections causes AttributeError: release_list
  • [PICARD-663] – Artist name makes it impossible to save

Improvement

  • [PICARD-658] – Support the new pregap and data tracks
  • [PICARD-659] – Set the originalyear tag when loading a release
  • [PICARD-665] – Web service calls to ports 80 and 443 do not need explicit port specification. 443 should be automatically made https.

Schema change upgrade instructions, schema 21

This upgrade shouldn’t be substantially different than past upgrades, now that we’ve fixed a few bugs with the process. To upgrade:

  1. Make sure your REPLICATION_TYPE setting is RT_SLAVE and your DB_SCHEMA_SEQUENCE is set to 20 in lib/DBDefs.pm.
  2. Ensure you’ve replicated up to the most recent replication packet available with the old schema. (if you’re not sure, run ./admin/replication/LoadReplicationChanges and see what it tells you).
  3. Take down the web server running MusicBrainz, if you’re running a web server.
  4. Turn off cron jobs if you are automatically updating the database via cron jobs.
  5. Switch to the new code with git fetch origin followed by git checkout schema-change-20-to-21
  6. Run ./upgrade.sh (or carton exec -Ilib -- ./upgrade.sh if you’re using carton, with very old setups).
  7. Set DB_SCHEMA_SEQUENCE to 21 in lib/DBDefs.pm
  8. Turn cron jobs back on, if needed.
  9. Restart the MusicBrainz web server, if applicable. It’s also recommended you restart memcached.

That’s it! The only real difference from the past is the specific tag to be used: schema-change-20-to-21, which is a couple of fix-up commits past the regular release tag.

Picard 1.3 released

Version 1.3 of Picard has been released today, with some notable features and bug fixes.

This release has few visible changes, but overall performance and stability were much improved. A lot of minor annoying issues were fixed. Cover art code was reworked, and notably one can now enable fallback on release group cover art if no cover art exists for a specific release.

Logging was improved, user can now enable debug mode from About →View Error/Debug Log, it may help to see what is going on if needed, passwords and sensible information were hidden from the log, so user can now safely post his log to get help.

To report any issue concerning this release, please use our bug tracker. As usual you can also get help on forum or on IRC (freenode, #musicbrainz).

Special thanks to Sophist, Wieland Hoffmann, Michael Wiencek, Johannes Dewender, Lukáš Lalinský, Frederik “Freso” S. Olesen, and the whole MusicBrainz Team.
Many thanks to all users, developers, bug reporters and translators. Code contributions since 1.2 are visible on Github Contributors graph.

Unfortunately the OAuth support patch (PICARD-305) didn’t make it in this release.

Picard 1.3 can be downloaded on the Picard’s website.

Be aware that downgrading from 1.3 to 1.2 may lead to configuration compatibility issues, better save your configuration before installing 1.3 if you intent to go back to 1.2.

Among changes since 1.2:

  • The “About” window now displays the versions of libraries used by Picard
  • Picard now correctly handles matching of MP3 files saved in ID3v2.3 tags (which is the version that Microsoft Windows and iTunes both use).
    Note: You may need to re-save your tags once to get them to match in future.
  • A sort tags plugin is now provided as tag data is no longer displayed sorted by default.
  • A new tag, musicbrainz_releasetrackid, containing the MusicBrainz Track MBID introduced in the May 2013 schema change release, is now written to files.
  • Add %_recordingtitle% (PICARD-515)
  • Fix plugin install bugs (PICARD-444)
  • Fix Options → File naming examples to handle primary/secondary release types (PICARD-516)
  • A new advanced option is available to permanently set the starting directory for the file browser and “Add files/folder” buttons.
  • Requests to Musicbrainz against your own account e.g. for collections are now handled through SSL (PICARD-337)
  • Refresh of Albums using Ctrl-R and selection of Other Releases are now more responsive during batch lookups.
  • Main window is now emitting a “selection_updated” signal, plugin api version bumps to 1.3.0
  • Append system information to user-agent string
  • Compilation tag/variable functionality (for tagging & file naming) has been split into two:
    – %compilation% is now aligned with iTunes, and set only for Various Artists type compilations
    – %_multiartist% variable now indicates whether this release has tracks by multiple artists
    (in order to prepend the artist name to the filename as shown in the default file naming script)
  • autodetect the CD drive on Mac OS X (PICARD-123)
  • Ignore directories and files while indexing when show_hidden_files option is set to False (PICARD-528)
  • Add ignore_regex option which allows one to ignore matching paths, can be set in Options → Advanced (PICARD-528)
  • Added an “artists” multi-value tag to track metadata, based on the one in Jaikoz, which contains the individual artist names from the artist credit. Also useful in scripts (joining phrases like ‘feat:’ are omitted) and plugins.
  • Added “_artists_sort“, “_albumartists“, “_albumartists_sort” variables for scripts and plugins.
  • Made Picard use the country names also used on the MusicBrainz website (PICARD-205)
  • New setup.py command `get_po_files` (Retrieve po files from transifex)
  • New setup.py command `regen_pot_file` (Regenerate po/picard.pot)
  • New Work tag (which for Classical music is often different from the track title) saved as ID3 TOAL tag.
  • New Composer Sort Order tag (variable %composersort%).
  • Improve the Other Releases list to prioritise and separate releases which match the correct number of tracks and your Options → Metadata → Prefered Releases settings for Country and Format.
  • New %_absolutetracknumber% variable numbering tracks sequentially regardless of disc structure (so you can numbers tracks on multi-disc releases without a disc number)
  • Support dropping image directly from Google image results to cover art box
  • Add %_musicbrainz_tracknumber% to hold track # as shown on MusicBrainz release web-page e.g. vinyl/cassette style A1, A2, B1, B2
  • Show the ID3 version of the file in the Info… dialog (Ctrl-I) (PICARD-218)
  • Fixed a bug where Picard crashed if a MP3 file had malformed TRCK or TPOS tags (PICARD-112)
  • Add –files option to setup.py build_ui, used to force .ui to .py regeneration (PICARD-566)
  • New setup.py command `update_constants` (Regenerate countries.py and attributes.py)
  • Made Picard use release groups, medium formats and cover art types also used on the MusicBrainz website
  • Use MusicBrainz Server translations for release groups, medium formats and cover art types
  • Add checkbox to toggle debug at runtime in log/debug view dialog
  • Add a plugin to add Artist Official Homepage relationships to the website tag (ID3 WOAR tag)
  • Add integrated functions $eq_any, $ne_all, $eq_all, $ne_any, $swapprefix and $delprefix.
  • Add %_performance_attributes%, containing performance attributes for the work e.g. live, cover, medley etc.
  • Use $inmulti in file naming scripts i.e. …$if($inmulti(%_performance_attributes%,medley), (Medley),)
  • Add optional `priority` parameter to `register_album_metadata_processor()` and `register_track_metadata_processor()`
    Default priority is `PluginPriority.NORMAL`, plugins registered with `PluginPriority.HIGH` will be run first, plugins registered with `PluginPriority.LOW` will run last
  • Add Standardise Performers plugin to convert e.g. Performer [piano and guitar] into
    Performer [piano] and Performer [guitar].
  • Add support for release group cover art fallback (PICARD-418, PICARD-53)
  • Add a clear button to search box
  • Honour preferred release formats when matching AcoustIds (PICARD-631)
  • Prevent ZeroDivisionError in some rare cases (PICARD-630)

About next stable Picard release

Picard team decided to release a new stable version of Picard before the end of the summer.
To help us, advanced users, translators and developers are encouraged to:

Exact release date is not yet known, and features aren’t yet frozen, but the only important change we’ll try to get in before final release is OAuth support (PICARD-305), we still accept minor patches until feature freeze is announced.

Since Picard is now using some translation resources from MusicBrainz project, translators should make sure following resources are up-to-date:

A simplified list of changes made since 1.2 can be read there.

Be aware that downgrading from 1.3 to 1.2 may lead to configuration compatibility issues, better save your configuration before using 1.3 branch if you intent to go back to 1.2.

 

 

Fire damages the Internet Archive

A fire at the Internet Archive (our friends!) has caused $600,000 in damage. Fortunately no one was harmed and no data was lost:

A fire at the Internet Archive’s San Francisco scanning center has destroyed an estimated $600,000 of digitization and scanning equipment. Fortunately no one was injured in the blaze, but the property damage has ruined “some physical materials” that were yet to be digitized, and restricted the nonprofit organization’s ability to record the history of the web.

MusicBrainz just donated $50 to the Internet Archive and asks you to consider making a donation as well.

Picard 1.2 released

Version 1.2 of Picard has been released today, with some notable features and bug fixes.

As announced recently, PUID support is being removed from MusicBrainz on October 15. This release of Picard has followed suit by removing all support for AmpliFIND/PUID scanning and submission. AcoustID is now the default (and only) method of scanning files. If you’re currently using AmpliFIND as your fingerprinting system, then after installing Picard 1.2 you must enable AcoustID in your options, under the “Fingerprinting” section.

Those who have the Picard cover art plugin installed will likely want to remove it. Picard 1.2 has built-in support for downloading cover images from a variety of sources, without the need for a plugin. Please see the new configuration under Options -> Cover Art.

Something important to note for people who build/package Picard is that Python 2.6 is now the minimum required version.

Special thanks to Wieland Hoffmann, Laurent Monin, Lukáš Lalinský, and everyone who’s helped contribute/report bugs for this release!

Picard 1.2 can be downloaded at:
http://musicbrainz.org/doc/MusicBrainz_Picard

Changes since 1.1:

  • Picard now requires at least Python 2.6
  • Removed support for AmpliFIND/PUIDs
  • Add support for the Ogg Opus file format
  • It’s now possible to download cover images without any plugin. Cover Art Archive images can be downloaded by image type
  • Improved directory scanning performance
  • Prefer already-loaded releases of the same RG when matching files
  • Allow dropping new files onto specific targets
  • Add basic collections management support (PICARD-84)
  • Allow adding custom tags in the tag editing dialog (PICARD-349)
  • Fix replacing of Windows-incompatible characters (PICARD-393)
  • Save both primary and secondary release types (PICARD-240)
  • Handle errors from the AcoustID service better (PICARD-391)
  • Accept HTTPS URLs on drag-and-drop (PICARD-378)