We’re having a schema change release on May 17, mostly to make small changes that will make our queries more efficient, ensure better constraints, and make some hardcoded options editable without schema changes in the future. We are also upping the required versions of both Perl and Node.js to 5.30 and 16.0 respectively (see the “Minimum version requirements” section below.)
Here’s our list of tickets for the Spring 2021 schema change, with descriptions of what’s being changed:
Schema changes
- MBS-1424: Add a “first release date” field to recordings. A very popular request for years, this allows requesting the date of the first ever release a recording appeared on. This adds materialized tables
recording_first_release_dateandrelease_first_release_datewhich are updated via triggers whenever the earliest date changes. The change was released as an optional extension to the main MusicBrainz server schema on Dec 16, 2020, but it will be added to the main schema during this schema change. - MBS-10208: Allow merging collections. Users who decide two of their collections should be joined into a larger one should be able to do so without having to move all the entities in the collection manually. This requires adding a
editor_collection_gid_redirecttable (equivalent to other existingx_gid_redirecttables) to ensure the old collection links redirect to the one they have been merged into. - MBS-10566: Convert
allowed_series_entity_typeandallowed_collection_entity_typeto tables to allow for additions without schema changes. The constraintsallowed_series_entity_typeandallowed_collection_entity_typespecify which types of entity can be used in series and collections, respectively. As such, if we want to add the possibility to create series of artists, we need to modify the constraint during a schema change. For ease of use, we are moving the constraints to be their own tables instead, allowing us to update them as needed in the future outside of a schema change release. - MBS-10647: Add [no label] to
b_del_label_specialtrigger for labels. Theb_del_label_specialtrigger ensures that any attempt to remove a special purpose label fails. Currently it only checks the special case “Deleted label”, but since “[no label]” is also a special purpose label that should never be deleted, we will add its ID to the trigger check. - MBS-10821: Remove orphaned recordings from collections for deletion. Replaces a single function,
delete_orphaned_recordings(), to add a new clause that makes it so that recordings referenced only in collections (but not linked to anything else in the database) can be deleted as orphans. This was released on the main MusicBrainz servers on June 15, 2020, but it will be added to the main schema during this schema change. - MBS-10962 / MBS-11460 – Add materialized tables and indexes to fetch releases and release groups by artist or track artist. These tickets will address performance issues on our current artist pages. They do not modify any existing tables, but as mentioned, add some new tables (to be updated via triggers) and indexes.
- MBS-11097: Support PKCE (Proof Key for Code Exchange) by OAuth clients. Adds two new columns to the
editor_oauth_tokentable. This feature is opt-in, but allows public OAuth2 clients to mitigate auth code interception attacks. The change was released as an optional extension to the main MusicBrainz server schema on Sep 21, 2020, but it will be added to the main schema during this schema change. - MBS-11431: Speed up
/ws/js/check_duplicates. Adds new indexes only (on the artist, label, place, and series tables, plus their respective alias tables). Improves some slow queries in the editing interface related to duplicate checking, i.e. finding other entities with the same name. Since this is a non-breaking change, it was released on the main MusicBrainz servers on Mar 15, 2021, but it will be added to the main schema during this schema change. - MBS-11451: Support ratings for places. Places can be reviewed in CritiqueBrainz yet cannot be rated in MusicBrainz. This is a strange state of affairs: clearly, if they are worth reviewing in depth, they also deserve the option to rate them. As such, we will add
place_rating_rawandplace_metatables, in the same way we have for other ratable entities. - MBS-11453: Change
entity0_cardinality,entity1_cardinalitytoSMALLINT. The cardinality columns of thelink_typetable are used to indicate whether the entity on each side of the relationship is expected to have only a few uses of the relationship type in question, or many (too many to comfortably display/edit). This is what stops every single recording of a work showing up in an edit work page, for example. At the moment we allow only two values for cardinality, 0/1. While it is possible that we will want to allow a few other values in between what is effectively “do not show anywhere ever” and “show all the time”, it is clear we will not need more than 32.000 values. As such, we are moving these columns fromINTEGERtoSMALLINTto reduce table size. - MBS-11456: Add MBIDs and redirect tables for artist credits. Adds a
gidcolumn to theartist_credittable, and a newartist_credit_gid_redirecttable. The MBIDs will allow public identification of artist credits outside of MusicBrainz, and open the door to useful features in the future. - MBS-11457: Drop series
ordering_attribute. This column was added back when we were expecting to have different types of ordering attributes for series, but we have never used it. We are planning to just drop the column. - MBS-11459: Add a script to create
edit_data_type_info.edit_data_type_infois a function used for development that we added in 2020 and we will now add to mirrors as well, both for consistency and for anyone who uses their mirror for development and just wants to use it. - MBS-11463: Add view to easily access medium track lengths. Our current way of finding the length of a medium is to either load all its tracks, then sum the durations, or to use the duration of its disc ID when present. The first of these requires a lot more processing than just getting the durations straight from the database, while the second ignores any data tracks not on the CD TOC. This adds a
medium_track_durationsview that allows easy access to the duration of the track lengths for any medium. - MBS-11464: Drop
statistics.log_statistic. We added thestatistics.log_statistictable for a Google Summer of Code project back during a 2012 schema change, but the work never got finished and implemented. We are not planning to implement it anymore, so this table is useless and we will be dropping it. - MBS-11466: Change
language.frequencyandscript.frequencytoSMALLINT. Thefrequencycolumn of a language or script indicates how often it’s used and lets us sort the most frequently used entries at the top of our lists. But we don’t store an exact count, just a number from 0-4 indicating “frequently used,” “hidden,” or “other.” Like MBS-11453 above, we don’t expect to need a fullINTEGERtype to store these columns at any point in the future, so can safely move them toSMALLINT.
Minimum version requirements
We’re raising the minimum required version for Node.js to v16 (which is the next LTS release coming in April 2021). Our current required version, v10, is hitting the end of its life cycle in April, and given there shouldn’t be a particular difficulty installing the current Node.js LTS on any system, it makes sense to just upgrade to the most recent LTS by the time of the schema change day.
We’re also raising the minimum required version for Perl to 5.30. The latest version is 5.32, but the current Ubuntu LTS (20.04), which is likely to be the next base image for our Docker containers, only provides 5.30. Our current required version, 5.18, was released all the way back in 2013, so moving to 5.30 is already a fairly significant improvement.
We’ll post final details about this release just prior to the release and shortly after we complete it, including instructions on how you can update your own copy. If you have any questions, please do leave a comment below or on the linked JIRA tickets!