MusicBrainz database schema change release, 2025-05-19 (with upgrade instructions)

We’re happy to announce the release of our May 2025 database schema change now! Thanks to all who were patient during Monday’s downtime as we released everything to our production servers.

This is a fairly small database schema change release which mostly makes minor improvements and fixes small bugs. Of the schema change tickets, the one you are most likely to notice while you browse and edit in MusicBrainz is MBS-9253 (which makes it possible to change the order in which we display release groups, and mean EPs now sort before singles). MBS-13832 means releases which have PDF front images now also show the cover art on the sidebar, and MBS-13768 provides permanent links to mediums.

Thanks to Brownd, salo.rock and silentbird for having reported bugs and suggested improvements, and thanks to BenjaBarcos1, Echelon, Joax, Marc Riera, Michal77, onemanue, ozodbekmirzayev and salo.rock for updating the translations.

A new release of MusicBrainz Docker is also available that matches this update of MusicBrainz Server. See the release notes for update instructions.

P.S. Search upgrades occurred separately ten days later.

Continue reading “MusicBrainz database schema change release, 2025-05-19 (with upgrade instructions)”

Schema change release: May 19, 2025

MusicBrainz is announcing a new schema change release set for May 19, 2025. Erratum: Search upgrades will shortly follow it. Like most of our recent schema changes, it should have little or no impact to downstream users.

There is one change to a major replicated table worth mentioning upfront: the medium table will have a new gid column added. If you’re running custom SQL queries against the database that join the medium table at all, there is a small chance you could run into errors like ERROR: column reference "gid" is ambiguous if you’re not properly qualifying the columns being selected.

We’re also altering some columns on the artist_release and artist_release_group tables (see below for more details). These are materialized tables used by our website on the back-end to speed up certain pages; you should normally not be accessing them directly, but it’s worth mentioning just in case. These tables do exist on mirrors, but are only populated with data if you’ve run admin/BuildMaterializedTables before.

Besides replacing some functions/triggers, you generally shouldn’t have to worry about any other database breaking changes in this release.

Finally, here is the complete list of scheduled tickets:

Database schema

The following tickets change the database schema in some way.

  • MBS-9253: List EP release groups above singles on artist pages. A small change to the get_artist_release_group_rows function is required in order to be able to change the sorting of release groups to prioritize EPs over singles. The function will be changed to depend on the type’s child_order (which can be safely changed at any time) rather than its id for sorting. While this function exists on mirrors, the function change shouldn’t have any impact on them directly (but a change of the child_order of the types will affect the sorting for display on mirrors as well). We’ll be adding new triggers to the release_group_primary_type and release_group_secondary_type tables to run the function when the tables change – these triggers will also exist on mirrors.
  • MBS-13322: Race condition when removing unused URLs. A rare internal error can occur in one of our trigger functions that cleans up unused URLs. We’ll replace that function, delete_unused_url, updating it to avoid a “race condition” whereby a URL can become used again the moment before it’s deleted. This will have no impact on mirrors, as delete_unused_url is only invoked by triggers that don’t exist on mirrors.
  • MBS-13464: Inconsistent sorting of artist release/release group titles. In the May 2021 schema change, we added some new materialized tables to significantly speed up the loading of artists’ release and release group listings: the not-so-surprisingly named artist_release and artist_release_group tables. These work by efficiently indexing an artist’s releases and release groups by date and other attributes, and then finally by their titles. Except for efficiency reasons, we originally decided to only store the first character of the titles for sorting. That predictably leads to incorrect sorting in certain cases, like with undated live bootlegs, as shown in MBS-13464. After measuring the actual size impact, we’ve decided to update the artist_release and artist_release_group tables to replace their sort_character columns with name columns that store the complete titles.
  • MBS-13768Add MBIDs to mediums. Adds a gid column to the medium table, and a new medium_gid_redirect table. It generates MBIDs for existing mediums that will be replicated to mirrors.
  • MBS-13832: Also support PDF files in CAA / EAA index_listing (for is_front purposes). PDF files are never treated as front for cover art archive purposes, probably because they originally did not have PNG thumbnails generated by the Internet Archive. That changed quite a while ago though, and there seems to be no reason to single them out anymore. We will just replace the index_listing views for cover_art_archive and event_art_archive with ones amended to not filter out PDF files.

    Note: Databases created before schema 25 (2017) may be missing filesize columns from their cover_art_archive.index_listing view (MBS-14014). This upgrade will add those columns. Since this view is for internal use, and these columns already exist on databases created in the past 8 years, we believe this shouldn’t pose any real compatibility issue.
  • MBS-13964: Some recordings are missing a first release date. A bug was discovered that causes recordings to sometimes have incorrect first-release-date values if any of the releases they’re attached to are merged with the “append” strategy. We’ll be adding a new trigger to the medium table that updates recording_first_release_date properly when such merges occur. Note that since recording_first_release_date is a materialized table, this trigger will also run on mirrors; that way it’s kept up-to-date even after running admin/BuildMaterializedTables initially.
  • MBS-13966: Release group first release dates need to be recalculated. Another (unrelated) issue with “first release date” information, but this time with release groups rather than recordings. We’ve found that a very small percentage of release groups’ first release dates (as stored in the release_group_meta table and returned in the web service) are wrong; it’s uncommon, but one way this can occur is when all releases in a release group are moved out of it. To address this, we’ll update the set_release_group_first_release_date function, which will have the exact same signature as before. We’ll also run a script to rebuild the incorrect data.

Erratum: This list originally included MBS-13965: Extend entity attribute schema to mediums – until we realized this was actually already done eight years ago. Isn’t time weird.

Update 2025-05-02: MBS-13966 was moved to this section, because we found out that a function needs to be updated to fix the underlying issue.

Update 2025-05-08: MBS-14014 has been fixed and mentioned as part of MBS-13832 above.

Search indexes

Erratum: Solr 9 will be made available for mirrors in a separate release.

Data corrections to the recording_first_release_date and release_group_meta tables do affect indexed recording and release group data respectively. If you have live search indexing enabled, those changes should be propagated to the search indexes automatically. Otherwise, you will have to perform a full reindex of those entities’ search indexes.

We’ll post upgrade instructions for standalone/mirror servers on the day of the release. If you have any questions, feel free to comment below or on the relevant above-linked tickets.

MusicBrainz schema change release, 2024-05-13 (with upgrade instructions)

We’re happy to announce the release of our May 2024 schema change now! Thanks to all who were patient during yesterday’s downtime as we released everything to our production servers. Final testing for the mirrors also took some more time. Note that the initially announced upgrades for MusicBrainz search engine are just about to reach our beta website, and thus are postponed for mirrors too but can be expected for the few next releases.

This is a fairly small schema change release which mostly removes unused code and improves things behind the scenes. Of the schema change tickets, there are only two that will directly affect users as they browse and edit in MusicBrainz. The first (MBS-13421) adds genre collections, so that editors can for example make a list of their favorite genres. The second (MBS-13514) allows entering 6-digit label codes.

Additionally, derat gratified us with making aliases more visible in the website, and some other improvements, while yellowhatpro polished cover art priority.

Thanks to chaban, cyberskull, thomasegger, Toad King, and wileyfoxyx for having reported bugs and suggested improvements. Thanks to 0x1026, Anonymous, “ApeKattQuest, MonkeyPython”, AshPigeon, Autom, brtc, chaban, claybiockiller, danBLOO, darlingz, eigenric, Evergarden, GABG, hashflu, hesheng, julian45, kellnerd, LiarOnce, Philipp Wolfer, salo.rock, suming, Vac31., wileyfoxyx, x1ao4, xiaole_0714223, yangmouren, yyb987, and zemeles for updating the translations. And thanks to all others who tested the beta version!

A new release of MusicBrainz Docker is also available that matches this update of MusicBrainz Server. See the release notes for update instructions.

Continue reading “MusicBrainz schema change release, 2024-05-13 (with upgrade instructions)”

Schema change release: May 13, 2024

MusicBrainz is announcing a new database schema change release for May 13, 2024. The main change will be to upgrade from PostgreSQL 12 to 16. Even though it isn’t database-related, Perl will also be upgraded from 5.30 to 5.38. Those will become the minimum required versions. Other changes are mostly clean-up and refactoring, with the exception of support for genre collections and (addendum) 6-digit label codes.

A few weeks after, the search engine will also be upgraded from Solr 7 to 9. Search indexes will have to be rebuilt on mirrors, which takes some time. Gladsomely, it will allow us to implement search improvements again.

Continue reading “Schema change release: May 13, 2024”

MusicBrainz schema change release, 2023-05-15 (with upgrade instructions)

We’re happy to announce the release of our May 2023 schema change today! Thanks to all who were patient during today’s downtime as we released everything to our production servers, and thanks to CatQuest, jesus2099, and yindesu for creating tickets.

This is a fairly small schema change release which mostly removes unused code and improves things behind the scenes. Of the schema change tickets, there are only two that will directly affect users as they browse and edit in MusicBrainz.

The first (MBS-12800) makes it so that cancelled releases are ignored when calculating the first release dates for recordings and release groups; since something that was cancelled was by definition not released, it should not be used as a first release date.

The second (MBS-11312) lays the foundation for a new feature (MBS-4685) that will allow users to edit or delete their edit notes, as long as a set of conditions are met (see the edit note docs for details). Admins will be able to edit or delete any edit notes at any time (MBS-13084), mostly to get rid of spam or offensive content, but also to help any editor who cannot change their note anymore but has an important reason why they need to do so. This feature will be available for testing in our beta server on Tuesday and we expect to release it next week, assuming our beta testers don’t find any too big issue with it by then.

A new release of MusicBrainz Docker is also available that matches this update of MusicBrainz Server. See the release notes for update instructions.

Continue reading “MusicBrainz schema change release, 2023-05-15 (with upgrade instructions)”

Schema change release: May 15, 2023

MusicBrainz is announcing a new schema change release for May 15, 2023. The actual schema (database) changes we’ve detailed below shouldn’t have much perceivable impact on mirror servers, especially if you only use the web service; most are to remove unused tables/columns or tweak how certain tables are materialized.

Continue reading “Schema change release: May 15, 2023”

Steps to fix missing genre and tag data on MusicBrainz mirror servers

If you recently updated your mirror server to the 2022-05-16 schema change release, we’re sorry to say that a bug in our upgrade script caused aggregate genre and tag data (if you had imported any) to be deleted. If you need this data, it can be re-imported from a recent dump, and we’ve written a script to help automate that.

You can safely ignore this post if

To restore the genre and tag data, follow these steps:

  1. Ensure you’ve replicated up to the most recent replication packet available. If you’re not sure, run ./admin/replication/LoadReplicationChanges. If you’re up-to-date, it should log “Replication packet … is not available.”
  2. Run git checkout production && git pull origin production.
  3. Turn off any cron jobs that update the database, including for replication.
  4. Run ./admin/sql/updates/20220720-mbs-12508.sh.
  5. Restart any cron jobs that you disabled.

You can verify that this process worked by checking the number of tags in the database: echo 'SELECT count(*) FROM tag' | ./admin/psql READWRITE. It should be over 200,000.

Sorry for the inconvenience, and let us know if you encounter any further issues.

MusicBrainz schema change release, 2022-05-16 (with upgrade instructions)

We’re happy to announce the release of our May 2022 schema change today! Thanks to all who were patient during today’s downtime as we released everything to our production servers, and thanks to ikerm2003, mfmeulenbelt rinmon and salo.rock for updating the translations.

This is once again a fairly minor release as far as schema changes go, but please do report any issues that you come across, especially related to the propagation of ratings and tags.

Continue reading “MusicBrainz schema change release, 2022-05-16 (with upgrade instructions)”

Schema change release: May 16, 2022

Today we’re announcing a MusicBrainz database schema change release planned for May 16, 2022. The majority of these changes follow the theme of improving data integrity and consistency, performance, or just cleaning up old cruft. Others relate to new features for genres and artist credits. We’re also introducing a new entity based on tags, like genres that came before it: Mood. See below for more details, including information on how these changes affect the schema or existing data. We expect people will encounter zero breaking changes, but it doesn’t hurt to double check, especially if you have a specific or non-standard use of the database!

Here’s our list of tickets for the Spring 2022 schema change:

Continue reading “Schema change release: May 16, 2022”

MusicBrainz schema change release, 2021-05-17 (with upgrade instructions)

We’re happy to announce the release of our May 2021 schema change today! Thanks to all who were patient during today’s downtime as we released everything to our production servers.

This is a fairly minor release as far as schema changes go, but please do report any issues that you come across, especially related to the display of recordings, releases and release groups on artist and release group pages.

New, user-facing changes with this release are limited to the new ability to merge collections (MBS-10208) and the addition of ratings for places (MBS-11451). Additionally, MBS-11463 adds a new view that is used to fix a couple small requests related to disc IDs (MBS-11268) and release length calculation (MBS-11349). Two other changes – adding a first-release-date field to recordings (MBS-1424) and support for PKCE in OAuth (MBS-11097) are more or less end-user affecting but were already released on the main MusicBrainz servers a while ago. All other changes are under the hood only.

We ran into a few complications while working on this schema change update, so we decided to postpone two changes to our October schema change to ensure only stuff we are more confident on is released. Those are MBS-11457, which involves dropping the ordering_attribute column for series and would have had no direct effect on user experience, and MBS-11456, which would have added MBIDs for artist credits.

A few of the released new features and improvements — namely the first-release-date field for recordings, and the performance improvements to artist pages — make use of new materialized tables. These tables aren’t dumped, nor are they replicated, since they’re derived entirely from primary table data. Rather, we’ve added a new script to build them (admin/BuildMaterializedTables, included in the upgrade instructions below), and triggers to keep them up-to-date once they’re built. These triggers are created on replicated servers, too. If you use the web interface or web service at all, just note the extra step of running BuildMaterializedTables after upgrade.sh below!

A new release of MusicBrainz Docker is also available that solves an issue for live indexing and matches this update of MusicBrainz Server. See the release notes for update instructions.

Now, on to the instructions.

Schema Change Upgrade Instructions

Note: Importing the latest data dump is always a valid alternative to running ./upgrade.sh on an existing database, if you’d prefer to also get new data in one go. Just follow the relevant instructions in INSTALL.md. The git tag is v-2021-05-19-hotfixes. The rest of the instructions here assume an in-place upgrade.

  1. Make sure DB_SCHEMA_SEQUENCE is set to 25 in lib/DBDefs.pm.
  2. If you’re using the live data feed (your REPLICATION_TYPE is set to RT_SLAVE), ensure you’ve replicated up to the most recent replication packet available with the old schema. If you’re not sure, run ./admin/replication/LoadReplicationChanges and see what it tells you; if you’re ready to upgrade, it should say “This replication packet matches schema sequence #26, but the database is currently at #25.”
  3. Take down the web server running MusicBrainz, if you’re running a web server.
  4. Turn off cron jobs if you’re automatically updating the database via cron jobs.
  5. If you’re using the live search indexing, stop it and, assuming sir is under the same directory as musicbrainz-server, run cd ../sir && python2.7 -m sir triggers && cd - && ./admin/psql < ../sir/sql/DropTriggers.sql && ./admin/psql < ../sir/sql/DropFunctions.sql
  6. Switch to the new code with git fetch origin followed by git checkout v-2021-05-19-hotfixes.
  7. Install newer dependencies Perl 5.30 or later and NodeJS 16 according to install prerequisites.
  8. Run cpanm --installdeps --notest . (note the dot at the end) to ensure your perl-based dependencies are up to date.
  9. Run ./upgrade.sh (it may take a while to vacuum at the end).
  10. Set DB_SCHEMA_SEQUENCE to 26 in lib/DBDefs.pm as instructed by the output of ./upgrade.sh.
  11. If you’re using the web interface or web service, run ./admin/BuildMaterializedTables --database=MAINTENANCE all to build new materialized tables. These will take several additional gigabytes of spaces and be kept up-to-date automatically via triggers. For more information, see INSTALL.md.
  12. If you’re using the live search indexing, assuming sir is under the same directory as musicbrainz-server, run cd ../sir && git fetch origin && git checkout v2.1.0 && python2.7 -m sir triggers && cd - && ./admin/psql < ../sir/sql/CreateFunctions.sql && ./admin/psql < ../sir/sql/CreateTriggers.sql and rebuild indexes (by running cd ../sir && python2.7 -m sir reindex && cd -) then start it in watch mode (with cd ../sir && git fetch origin && git checkout v2.1.0 && python2.7 -m sir amqp_watch)
  13. Turn cron jobs back on, if applicable.
  14. Restart the MusicBrainz web server, if applicable. It’s also recommended you restart Redis. If you’re accessing your MusicBrainz server in a web browser, run ./script/compile_resources.sh.

Here’s the list of resolved tickets:

New Feature

  • [MBS-10208] – Allow merging collections
  • [MBS-11451] – Support ratings for places
  • [MBS-11463] – Add view to easily access medium track lengths
  • [MBS-11652] – Add support for artist series (hotfixed)

Improvement

  • [MBS-10962] – Speed up listing artist’s releases
  • [MBS-11268] – Show “Set track durations” on release/discids page
  • [MBS-11460] – Add materialized tables to fetch release groups by artist or track artist

Database Schema Change Task

  • [MBS-10647] – Add [no label] to b_del_label_special trigger for labels
  • [MBS-11453] – Change entity0_cardinality, entity1_cardinality to SMALLINT
  • [MBS-11459] – Create the edit_data_type_info function on mirrors
  • [MBS-11464] – Drop table statistics.log_statistic
  • [MBS-11466] – Change language.frequency and script.frequency to SMALLINT

Previously Released Changes

  • [MBS-1424] – Add a ‘First release date’ field to recordings
  • [MBS-10821] – Edit changing medium tracklist and format is stuck
  • [MBS-11097] – Support PKCE (Proof Key for Code Exchange) by OAuth clients
  • [MBS-11431] – Speed up /ws/js/check_duplicates