MusicBrainz schema change release, 2022-05-16 (with upgrade instructions)

We’re happy to announce the release of our May 2022 schema change today! Thanks to all who were patient during today’s downtime as we released everything to our production servers, and thanks to ikerm2003, mfmeulenbelt rinmon and salo.rock for updating the translations.

This is once again a fairly minor release as far as schema changes go, but please do report any issues that you come across, especially related to the propagation of ratings and tags.

New, user-facing changes with this release include withdrawn-only release groups showing in the official overview again (MBS-12208) and the final disappearance of Amazon cover art (MBS-12200). To this regard, the report of releases that have Amazon cover art but no Cover Art Archive front cover will stay available for editors to check their subscribed entities.

Additionally, several small changes will allow, in the next couple of releases, to store more information about genres (including URL relationships) and to recognize and special-case mood tags (MBS-12190). Another new feature that will start to be used in the API and artist credit pages is the addition of MusicBrainz IDs for artist credits, which allow referring to them with a unique and more persistent ID (MBS-11456). Finally, a few more under-the-hood only changes are made, which should ensure better performance for finding artists, events, etc. for all areas contained in a given one, and less bugs when adding and changing tags and ratings.

The area containment changes make use of a new materialized table. Like the ones we added last year, this table isn’t dumped nor replicated, since it is derived entirely from primary table data. Rather, it will be created during migration (or, in a new install, by running the admin/BuildMaterializedTables script) and triggers will be added to keep it up-to-date once it has been built. These triggers are created on replicated servers, too.

The accompanying new version of the search index rebuilder brings performance improvements for both the main server and mirrors, and simplifies maintenance. See the release notes for details.

A new release of MusicBrainz Docker is also available that matches this update of MusicBrainz Server. See the release notes for update instructions.

Now, on to the instructions.

Schema Change Upgrade Instructions

Note: Importing the latest data dump is always a valid alternative to running ./upgrade.sh on an existing database, if you’d prefer to also get new data in one go. Just follow the relevant instructions in INSTALL.md. The git tag is v-2022-05-16-schema-change. The rest of the instructions here assume an in-place upgrade.

  1. Make sure DB_SCHEMA_SEQUENCE is set to 26 in lib/DBDefs.pm.
  2. If you’re using the live data feed (your REPLICATION_TYPE is set to RT_SLAVE), ensure you’ve replicated up to the most recent replication packet available with the old schema. If you’re not sure, run ./admin/replication/LoadReplicationChanges and see what it tells you; if you’re ready to upgrade, it should say “This replication packet matches schema sequence #27, but the database is currently at #26.”
  3. Take down the web server running MusicBrainz, if you’re running a web server.
  4. Turn off cron jobs if you’re automatically updating the database via cron jobs.
  5. If you’re using the live search indexing, stop it and, assuming sir is under the same directory as musicbrainz-server, run cd ../sir && python2.7 -m sir triggers && cd - && ./admin/psql < ../sir/sql/DropTriggers.sql && ./admin/psql < ../sir/sql/DropFunctions.sql
  6. Switch to the new code with git fetch origin followed by git checkout v-2022-05-16-schema-change.
  7. Run cpanm --installdeps --notest . (note the dot at the end) to ensure your perl-based dependencies are up to date.
  8. Run ./upgrade.sh (it may take a while to vacuum at the end).
  9. Set DB_SCHEMA_SEQUENCE to 27 in lib/DBDefs.pm as instructed by the output of ./upgrade.sh.
  10. If your REPLICATION_TYPE is set to RT_SLAVE, change it to RT_MIRROR. (The previous terminology will work for the time being, but is now deprecated.)
  11. If you’re using the live search indexing, assuming again that sir is under the same directory as musicbrainz-server, run cd ../sir && git fetch origin && git checkout v3.0.1 && python2.7 -m sir triggers && cd - && ./admin/psql < ../sir/sql/CreateFunctions.sql && ./admin/psql < ../sir/sql/CreateTriggers.sql and rebuild indexes which takes hours (by running cd ../sir && python2.7 -m sir reindex && cd -) then start it in watch mode (with cd ../sir && git fetch origin && git checkout v3.0.1 && python2.7 -m sir amqp_watch)
  12. Turn cron jobs back on, if applicable.
  13. Restart the MusicBrainz web server, if applicable. It’s also recommended you restart Redis. If you’re accessing your MusicBrainz server in a web browser, run ./script/compile_resources.sh.

Here’s the list of resolved tickets:

Fixed Bug

  • [MBS-5359] – *_tag tables are corrupt and need to be regenerated
  • [MBS-11760] – Removing the last use of a tag does not always remove the tag
  • [MBS-12369] – Standalone databases may be missing foreign keys for the documentation schema

New Feature

  • [MBS-12190] – Add Mood support in the database

Improvement

  • [MBS-11456] – Add MBIDs to artist credits in the database with merge
  • [MBS-12141] – Block tag names that are empty or have uncontrolled whitespace with database constraints
  • [MBS-12224] – Keep tags’ ref_count and aggregate vote counts updated with triggers
  • [MBS-12249] – Add a materialized area_containment table kept up-to-date with triggers
  • [MBS-12256] – Keep rating and rating_count column on *_meta tables up-to-date with triggers
  • [MBS-12313] – Clarify item naming in the Search drop down menu

Database Schema Change Task

  • [MBS-11457] – Drop the series ordering_attribute column
  • [MBS-11755] – Remove unused tags
  • [MBS-12157] – Remove support for Amazon cover art
  • [MBS-12200] – Drop schema objects related to Amazon cover art support
  • [MBS-12225] – Rename “slave” to “mirror” (inclusive language update)
  • [MBS-12250] – Create dbmirror2 schema on production and mirror servers
  • [MBS-12252] – Add edit_genre table
  • [MBS-12253] – Add relationship tables for genres
  • [MBS-12254] – Add genre_annotation table
  • [MBS-12255] – Add genre_alias_type table and make genre_alias consistent

Picard 2.8 Release Candidate 2

We have decided to put out another release candidate Picard 2.8.0rc2 for the upcoming Picard 2.8. We got some valuable feedback and fixed some new bugs as well as some older ones that just got detected while testing the first release candidate. Thanks a lot to everyone who reported those issues.

This is a pre-release we put out for wider testing and to gather feedback on the changes before the final 2.8 release. Please report any issue through our bug tracker and give us feedback on this beta release on the Community Forums.

Download

Picard 2.8.0rc2 can be downloaded from the Picard website Downloads section.

Linux users might want to install the beta version using Snap. If your Linux distribution supports Snap you can install Picard from the beta channel using:

snap install --candidate picard

Bugfixes

  • [PICARD-2465] – RecursionError after dragging folder from file browser
  • [PICARD-2470] – CD Lookup error: utf-8 codec can’t decode byte 0xff in position 0: invalid start byte
  • [PICARD-2472] – Cluster action applies to already matched albums
  • [PICARD-2473] – CD disc ID from log files is unavailable on Windows if there is no CD drive
  • [PICARD-2474] – Revert PICARD-2420: Adding a file with existing AcoustID fingerprint and recording MBID in the tags activates submission
  • [PICARD-2475] – If search dialog query contains an MBID “Lookup in browser” will not open in browser

Helping out

The easiest way to help us getting a great Picard 2.8 release is using and testing this beta release. Please report bugs on the Picard issue tracker and provide feedback in the community forums.

Please also help translate Picard. There have been many changes to the user interface existing translations need to be updated for the final 2.8 release. Translating is easy and can be done online: Head over to MusicBrainz’s translation page on Transifex and click on “Help Translate MusicBrainz”.
Once you have registered an account on Transifex you can start translating. For Picard the primary resource to translate is “picard“, but there is also the “picard_appstream” resource, which is used for providing descriptions for various Linux software-center applications, and “picard_installer”, which contains the translations for Picard’s Windows installer.

If you are a software developer you are very welcomed to provide fixes and features. Picard is free software and the source code is available on GitHub. See Developing on the Picard website to get started.

Picard 2.8 Release Candidate

The Picard team is happy to announce the availability of the first release candidate for the upcoming Picard 2.8. This is a pre-release we put out for wider testing and to gather feedback on the changes before the final 2.8 release.

Please report any issue through our bug tracker and give us feedback on this beta release on the Community Forums.

Thanks a lot to everybody who contributed to this release with code, translations, bug reports and general feedback. This release contains code contributions by Philipp Wolfer, Bob Swift, Laurent Monin, jesus2099, Adam James, cybersphinx and Aerozol.

Download

Picard 2.8 Release Candidate can be downloaded from the Picard website Downloads section.

Linux users might want to install the beta version using Snap. If your Linux distribution supports Snap you can install Picard from the beta channel using:

snap install --candidate picard

What’s new?

Noteworthy changes in this release contain:

  • Support for file paths with more than 259 characters on Windows. Please note that not all Windows software is compatible with this, hence there is an option to disable long path support. See the documentation for details.
  • Ability to query disc IDs from EAC, XLD or Whipper log files, see the documentation.
  • Advanced relationships can now be loaded for releases with more than 500 tracks. Note that this requires additional requests to the MusicBrainz server (one per 100 tracks) and is hence slower than loading smaller releases.
  • AcousticBrainz support has been dropped. This follows the announcement of AcousticBrainz being discontinued.
  • Various improvements to “Submit cluster as release”

The Picard documentation has also been updated to reflect the changes in this release.

Below is a complete list of changes since Picard 2.7.3.

Bugfixes

  • [PICARD-1570] – Windows: Files with path length > 259 char fail to load
  • [PICARD-2292] – When a recording is a performance of multiple works, any instrumental performance attribute erases all other lyrics languages
  • [PICARD-2368] – Matching files compares “totaltracks” to the total no. of tracks across all mediums on the release
  • [PICARD-2398] – “Use track relationships” doesn’t work on large releases
  • [PICARD-2399] – Crash on login if options get closed before login requests finished
  • [PICARD-2417] – macOS: Expand/Collapse tracklist should use Left cursor to collapse
  • [PICARD-2420] – Adding a file with existing AcoustID fingerprint and recording MBID in the tags activates submission
  • [PICARD-2423] – Dragging album with multiple files matched to a track back to unmatched moves only half of the files
  • [PICARD-2425] – Picard crashes when accessing WinFSP / SSHFS-Win share
  • [PICARD-2430] – “Submit cluster as release” drops text after quotation marks
  • [PICARD-2435] – File naming examples crash if selected target directory exceeds Windows path limit
  • [PICARD-2440] – FLAC cover art does not preserve ID3 image type
  • [PICARD-2453] – “Submit cluster as release” won’t submit catalog number if there is no label tag
  • [PICARD-2454] – UI blocks when loading releases with a huge amount of tracks
  • [PICARD-2457] – “Search for similar albums” loads cover art into wrong rows
  • [PICARD-2459] – Crash if temporary cover art files are removed from disk
  • [PICARD-2461] – File does not show error icon after saving
  • [PICARD-2463] – Cover art box does not handle different screen scalings on multi-screen setups
  • [PICARD-2464] – Cover art stack on HiDPI displays rendered too small

New Features

  • [PICARD-1455] – Use EAC / XLD log files for disc ID lookup
  • [PICARD-2410] – Use Whipper log files for disc ID lookup
  • [PICARD-2411] – Add option to remove broken seektable from FLAC files

Tasks

  • [PICARD-2332] – Convert code to use scoped PyQt enums
  • [PICARD-2422] – Remove AcousticBrainz analysis and submission features

Improvements

  • [PICARD-993] – Better error icons for file loading / saving errors
  • [PICARD-2076] – Respect Windows 10 > 1607 removal of 260 limit on filenames
  • [PICARD-2282] – Provide ability to import and export Picard config
  • [PICARD-2349] – Use consistent terminology for “standalone recordings”
  • [PICARD-2369] – Renamed “Preferred release formats” to “Preferred medium formats”
  • [PICARD-2379] – Script options: Clarify what activating / deactivating scripts means
  • [PICARD-2380] – Update to fpcalc 1.5.1 for Windows / macOS packages
  • [PICARD-2382] – Make it easier to create a multi-value field value containing duplicate values
  • [PICARD-2392] – Allow multi-value variables to contain empty strings
  • [PICARD-2396] – Do not submit AcoustID fingerprints on significant track length mismatch
  • [PICARD-2402] – Make ‘end’ argument optional for $substr() function
  • [PICARD-2405] – Support CAA cover art type “Matrix/Runout”
  • [PICARD-2407] – Set browser integration port in add cluster as release functionality
  • [PICARD-2409] – Allow searching and dropping MusicBrainz disc ID URLs
  • [PICARD-2415] – Make “Other versions” easier to access as a separate dialog
  • [PICARD-2419] – Improve track number from filename detection to not treat e.g. UB40 at end of filename as track number
  • [PICARD-2421] – Consider track MBID when matching files to tracks
  • [PICARD-2437] – Tag editor dialog box should say “OK” instead of “Save”

Helping out

The easiest way to help us getting a great Picard 2.8 release is using and testing this beta release. Please report bugs on the Picard issue tracker and provide feedback in the community forums.

Please also help translate Picard. There have been many changes to the user interface existing translations need to be updated for the final 2.8 release. Translating is easy and can be done online: Head over to MusicBrainz’s translation page on Transifex and click on “Help Translate MusicBrainz”.
Once you have registered an account on Transifex you can start translating. For Picard the primary resource to translate is “picard“, but there is also the “picard_appstream” resource, which is used for providing descriptions for various Linux software-center applications, and “picard_installer”, which contains the translations for Picard’s Windows installer.

If you are a software developer you are very welcomed to provide fixes and features. Picard is free software and the source code is available on GitHub. See Developing on the Picard website to get started.