Author: reosarevok

MusicBrainz Server update, 2021-07-19

A week late, but here’s another server update! This time we fixed a bunch of bugs both new and old, added a bunch of new reports, and made it possible to find edits based on the content of their edit notes. Additionally, the bottom credits display for releases now hides irrelevant work relationships (they have been hidden in the inline display for a few months already).

A new release of MusicBrainz Docker is also available that matches this update of MusicBrainz Server. See the release notes for update instructions.

Thanks to cam1170, CatQuest, chaban, Cyberskull, jesus2099, ROpdebee and yindesu for having reported bugs and suggested improvements. Thanks to Besnik, kellnerd, mfmeulenbelt, salo.rock, yoshi818 and zas for updating the translations. And thanks to all others who tested the beta version!

The git tag is v-2021-07-19.

Fixed Bug

  • [MBS-11711] – Bad request for no longer existing paginated relationships
  • [MBS-11726] – Typo mistake in set cover art>Comment
  • [MBS-11753] – Artist credit editor range select is broken
  • [MBS-11757] – CDJapan sidebar link is misspelled as “CD Japan”
  • [MBS-11761] – Blank ratings in release collections pages
  • [MBS-11767] – Track-level artists that differ from the release artist are no longer shown on multi-disc releases that aren’t fully loaded
  • [MBS-11775] – Guess Feat is broken for release groups
  • [MBS-11776] – “loading-message” class missing from release loading section

Improvement

  • [MBS-10367] – Allow requesting copy of sent report when reporting
  • [MBS-11443] – Add option to sort edits by edit note date
  • [MBS-11578] – Link tracks in “Recordings with a different name than their only track” report
  • [MBS-11614] – Allow merging releases with the same structure when a medium is empty
  • [MBS-11664] – Enable Disc IDs tab when any are attached to unsupported mediums
  • [MBS-11744] – Improve Release date mouse selection (for copy)
  • [MBS-11747] – Consistent display of edit preview in release editor
  • [MBS-11765] – Add standalone recording statistics
  • [MBS-11769] – Display number of recordings with ISRCs and works with ISWCs on stats page
  • [MBS-11772] – Add the Greek (συμμ. <artist name>) to the Featured Artists reports
  • [MBS-11773] – Make “featuring in title” reports case insensitive
  • [MBS-11774] – Add fullwidth ｆｅａｔ / ｆｅａｔｕｒｉｎｇ to the featuring reports
  • [MBS-11782] – Also hide medleys, versions, etc. on the bottom credits view of releases

New Feature

  • [MBS-8371] – Search in edit notes
  • [MBS-10844] – Report for same URL linked to many entities
  • [MBS-11335] – Report for reused Wikidata items
  • [MBS-11381] – Add a way to add and update timeline events through the website
  • [MBS-11666] – Report for releases that have disc IDs but are not supposed to have them
  • [MBS-11679] – Wish users a happy MB anniversary

React Conversion Task

MusicBrainz Server update, 2021-06-28

This time we have a release with a fair amount of small bug fixes and improvements, but the main improvement you might notice is that release pages are now a lot more efficient for releases with a lot of tracks. Any release with more than 100 tracks will be loaded only partially, and more tracks can be loaded if useful. This avoids loading hundreds of tracks (and all relevant relationships) in cases where the user doesn’t actually need them, and it will make it possible to load some releases with over a thousand tracks which would consistently time out before.

A new release of MusicBrainz Docker is also available that matches this update of MusicBrainz Server. See the release notes for update instructions.

Thanks to yyoung for improving errors in URL editor. Thanks to chaban, Cyberskull, jaw111, Jeluang, jesus2099, nikki, salo.rock, Yurim for having reported bugs and suggested improvements. Thanks to Besnik, kellnerd, mfmeulenbelt and salo.rock for updating the translations. And thanks to all others who tested the beta version!

The git tag is v-2021-06-28.

Fixed Bug

  • [MBS-6811] – Edit note permalinks are not always canonical
  • [MBS-8678] – Split Artist not blocked by all relationships
  • [MBS-8844] – “Display Credits at Bottom” doesn’t work for releases with collapsed mediums
  • [MBS-11522] – Regression: Link to release group missing in “Set cover art” edit
  • [MBS-11697] – Old release group type (now secondary) not displayed on edits
  • [MBS-11703] – Release index pages with a lot of tracks and relationships often time out
  • [MBS-11724] – Typo in Disc IDs attached but not applied report
  • [MBS-11734] – ISE when filtering duplicate artists report
  • [MBS-11737] – The cover art upload form claims “we don’t own the associated item” even though we do

Improvement

  • [MBS-5632] – Make it clear in Set Cover Art edits that the old cover art is not displayed because it was automatically picked
  • [MBS-5680] – Allow inc=artist-credits when combined with release-group-rels, release-rels or recording-rels
  • [MBS-8700] – Include genres in JSON-LD (RDF) representation of resources
  • [MBS-9695] – Allow editing link_creditable_attribute_type / link_text_attribute_type from the UI
  • [MBS-9826] – Make edit expiration time show days + hours when it will take more than a day
  • [MBS-10307] – Cover art types selector are inconsistent between add and edit actions
  • [MBS-11342] – Clarify that recording duration will change if track duration is changed
  • [MBS-11421] – Give useful message when trying to split empty artist
  • [MBS-11644] – Allow specifying target release MBID when moving disc ID
  • [MBS-11698] – More specific target of external link error messages

New Feature

  • [MBS-8070] – Show Events in contained Places in Area Events tab
  • [MBS-8866] – Show Artist relations on Split Artist page

React Conversion Task

  • [MBS-9921] – Convert the release index page to React
  • [MBS-11520] – Convert Remove Relationship Type edit to React
  • [MBS-11523] – Convert Set Cover Art edit to React

Other Task

  • [MBS-10463] – Remove the iframe-based cover art image uploader
  • [MBS-10711] – Convert standard report lists to react-table
  • [MBS-11677] – Add Selenium tests for the CAA

MusicBrainz Server update, 2021-06-14

This week we spent some time looking through code that had been sitting for a long time waiting for a review while we worked on all sorts of other things. Most of the changes are small – one that might be more noticeable is that on recording lists you can now see the release groups the recording appears on. This should make it easier to figure out what is going on when a list has many recordings with the same name.

A new release of MusicBrainz Docker is also available that matches this update of MusicBrainz Server. See the release notes for update instructions.

Thanks to yyoung for fixing a display bug with external links. Thanks to chaban, HibiscusKazeneko, Ian Tresman, Jeluang, otringal and salo.rock for having reported bugs and suggested improvements. Thanks to ffff23 and salo.rock for updating the translations. And thanks to all others who tested the beta version!

The git tag is v-2021-06-14.

Fixed Bug

  • [MBS-11371] – JSON-LD “member” returns “1991-1992” only for “1991-1992, 2000-present”
  • [MBS-11440] – Subscribers page for private collection is not private
  • [MBS-11638] – Work form allows adding duplicate work attributes
  • [MBS-11639] – Changing work attribute type always keeps the old value
  • [MBS-11640] – Errors never shown for work attribute fields with fixed values
  • [MBS-11682] – Deadlock when entering edits
  • [MBS-11688] – The external link section will disappear once any link starts with //
  • [MBS-11700] – ISE when viewing “Remove disc ID” edit
  • [MBS-11702] – Disc ID: Same release displayed twice when moving disc-id

Improvement

  • [MBS-4826] – Display source release next to recordings
  • [MBS-10205] – Readd a link to the Data Removal Policy under the “About us” drop-down menu
  • [MBS-11293] – Report: streaming relationships in non-digital releases
  • [MBS-11337] – Always link to URL entity page and link on URL reports
  • [MBS-11609] – Show genre disambiguation on autocomplete dropdown
  • [MBS-11627] – Don’t show “delete” link for deleted editors
  • [MBS-11696] – Disallow Twitter documentation links

New Feature

  • [MBS-9098] – Add JSON-LD “memberOf” relationship to artists who are members of groups
  • [MBS-10107] – Add overview pages for relationship attributes
  • [MBS-11290] – Report of works with a type which are part of a work with the same type
  • [MBS-11305] – Report for erroneous looking trailing join phrases

React Conversion Task

  • [MBS-10997] – Convert /search/error files to React
  • [MBS-11326] – Convert edit/notes.tt to React
  • [MBS-11430] – Convert format_setlist to Javascript

Other Task

  • [MBS-11330] – Split root/types.js into smaller files
  • [MBS-11553] – Move from passing $c to using React.useContext in layout pages

MusicBrainz Server update, 2021-05-31

We’re back after our schema change with another release fixing mostly small bugs. The most welcome of these changes is probably that unrelated work relationships (such as arrangements, parodies, translated versions and whatnot) will now be hidden from the inline release view, meaning releases with popular songs that have lots of versions should be significantly less annoying to use!

A new release of MusicBrainz Docker is also available that matches this update of MusicBrainz Server. See the release notes for update instructions.

Thanks to chaban, fabe56, jesus2099, kellnerd, mr_maxis, mtrolley, psychoadept, ROpdebee and yindesu for having reported bugs and suggested improvements. Thanks to ffff23, jormangeud, outsidecontext and salo.rock for updating the translations. And thanks to all others who tested the beta version!

The git tag is v-2021-05-31.

Fixed Bug

  • [MBS-11630] – JS error when submitting parallel release group name edits from release editor
  • [MBS-11631] – DNB validation wrongly blocks label and place links
  • [MBS-11637] – Event-Work relationships not shown from Work page
  • [MBS-11643] – URL that ends in the future is prematurely disabled
  • [MBS-11660] – Periods are not always ignored in admin e-mail searches
  • [MBS-11662] – Guess Case changes capitalization of [untitled] to [Untitled] if not the only word on title
  • [MBS-11663] – Release group headings missing when moving disc ID
  • [MBS-11671] – Dates displayed as 24:mm instead of 00:mm
  • [MBS-11676] – Regression: “Set track durations” is shown for all releases on cdtoc pages

Improvement

  • [MBS-9419] – Don’t show versions, medleys, etc for a work on the release page
  • [MBS-11628] – Accept and cleanup https “permalinks” for DNB
  • [MBS-11673] – Block smart links: myurls.bio

New Feature

MusicBrainz schema change release, 2021-05-17 (with upgrade instructions)

We’re happy to announce the release of our May 2021 schema change today! Thanks to all who were patient during today’s downtime as we released everything to our production servers.

This is a fairly minor release as far as schema changes go, but please do report any issues that you come across, especially related to the display of recordings, releases and release groups on artist and release group pages.

New, user-facing changes with this release are limited to the new ability to merge collections (MBS-10208) and the addition of ratings for places (MBS-11451). Additionally, MBS-11463 adds a new view that is used to fix a couple small requests related to disc IDs (MBS-11268) and release length calculation (MBS-11349). Two other changes – adding a first-release-date field to recordings (MBS-1424) and support for PKCE in OAuth (MBS-11097) are more or less end-user affecting but were already released on the main MusicBrainz servers a while ago. All other changes are under the hood only.

We ran into a few complications while working on this schema change update, so we decided to postpone two changes to our October schema change to ensure only stuff we are more confident on is released. Those are MBS-11457, which involves dropping the ordering_attribute column for series and would have had no direct effect on user experience, and MBS-11456, which would have added MBIDs for artist credits.

A few of the released new features and improvements — namely the first-release-date field for recordings, and the performance improvements to artist pages — make use of new materialized tables. These tables aren’t dumped, nor are they replicated, since they’re derived entirely from primary table data. Rather, we’ve added a new script to build them (admin/BuildMaterializedTables, included in the upgrade instructions below), and triggers to keep them up-to-date once they’re built. These triggers are created on replicated servers, too. If you use the web interface or web service at all, just note the extra step of running BuildMaterializedTables after upgrade.sh below!

A new release of MusicBrainz Docker is also available that solves an issue for live indexing and matches this update of MusicBrainz Server. See the release notes for update instructions.

Now, on to the instructions.

Schema Change Upgrade Instructions

Note: Importing the latest data dump is always a valid alternative to running ./upgrade.sh on an existing database, if you’d prefer to also get new data in one go. Just follow the relevant instructions in INSTALL.md. The git tag is v-2021-05-19-hotfixes. The rest of the instructions here assume an in-place upgrade.

  1. Make sure DB_SCHEMA_SEQUENCE is set to 25 in lib/DBDefs.pm.
  2. If you’re using the live data feed (your REPLICATION_TYPE is set to RT_SLAVE), ensure you’ve replicated up to the most recent replication packet available with the old schema. If you’re not sure, run ./admin/replication/LoadReplicationChanges and see what it tells you; if you’re ready to upgrade, it should say “This replication packet matches schema sequence #26, but the database is currently at #25.”
  3. Take down the web server running MusicBrainz, if you’re running a web server.
  4. Turn off cron jobs if you’re automatically updating the database via cron jobs.
  5. If you’re using the live search indexing, stop it and, assuming sir is under the same directory as musicbrainz-server, run cd ../sir && python2.7 -m sir triggers && cd - && ./admin/psql < ../sir/sql/DropTriggers.sql && ./admin/psql < ../sir/sql/DropFunctions.sql
  6. Switch to the new code with git fetch origin followed by git checkout v-2021-05-19-hotfixes.
  7. Install newer dependencies Perl 5.30 or later and NodeJS 16 according to install prerequisites.
  8. Run cpanm --installdeps --notest . (note the dot at the end) to ensure your perl-based dependencies are up to date.
  9. Run ./upgrade.sh (it may take a while to vacuum at the end).
  10. Set DB_SCHEMA_SEQUENCE to 26 in lib/DBDefs.pm as instructed by the output of ./upgrade.sh.
  11. If you’re using the web interface or web service, run ./admin/BuildMaterializedTables --database=MAINTENANCE all to build new materialized tables. These will take several additional gigabytes of spaces and be kept up-to-date automatically via triggers. For more information, see INSTALL.md.
  12. If you’re using the live search indexing, assuming sir is under the same directory as musicbrainz-server, run cd ../sir && git fetch origin && git checkout v2.1.0 && python2.7 -m sir triggers && cd - && ./admin/psql < ../sir/sql/CreateFunctions.sql && ./admin/psql < ../sir/sql/CreateTriggers.sql and rebuild indexes (by running cd ../sir && python2.7 -m sir reindex && cd -) then start it in watch mode (with cd ../sir && git fetch origin && git checkout v2.1.0 && python2.7 -m sir amqp_watch)
  13. Turn cron jobs back on, if applicable.
  14. Restart the MusicBrainz web server, if applicable. It’s also recommended you restart Redis. If you’re accessing your MusicBrainz server in a web browser, run ./script/compile_resources.sh.

Here’s the list of resolved tickets:

New Feature

  • [MBS-10208] – Allow merging collections
  • [MBS-11451] – Support ratings for places
  • [MBS-11463] – Add view to easily access medium track lengths
  • [MBS-11652] – Add support for artist series (hotfixed)

Improvement

  • [MBS-10962] – Speed up listing artist’s releases
  • [MBS-11268] – Show “Set track durations” on release/discids page
  • [MBS-11460] – Add materialized tables to fetch release groups by artist or track artist

Database Schema Change Task

  • [MBS-10647] – Add [no label] to b_del_label_special trigger for labels
  • [MBS-11453] – Change entity0_cardinality, entity1_cardinality to SMALLINT
  • [MBS-11459] – Create the edit_data_type_info function on mirrors
  • [MBS-11464] – Drop table statistics.log_statistic
  • [MBS-11466] – Change language.frequency and script.frequency to SMALLINT

Previously Released Changes

  • [MBS-1424] – Add a ‘First release date’ field to recordings
  • [MBS-10821] – Edit changing medium tracklist and format is stuck
  • [MBS-11097] – Support PKCE (Proof Key for Code Exchange) by OAuth clients
  • [MBS-11431] – Speed up /ws/js/check_duplicates

MusicBrainz Server update, 2021-05-03

Hi! This time we have another small release with a bunch of minor fixes and small quality of life improvements. The latter include one small improvement to an error message requested over 10 years ago! It might take us time, but we get to things eventually, and that’s what matters, right? Right? Next release will be our May schema change – you can read what is coming with that one at the dedicated announcement post.

A new release of MusicBrainz Docker is also available that solves an issue for live indexing and matches this update of MusicBrainz Server. See the release notes for update instructions.

Thanks to chaban, Cyberskull, Lotheric, nikki, salo.rock and Sophist for having reported bugs and suggested improvements. Thanks to mfmeulenbelt, outsidecontext and salo.rock for updating the translations. And thanks to all others who tested the beta version!

The git tag is v-2021-05-03.

Fixed Bug

  • [MBS-11550] – Untranslated strings on Confirm Form Submission page
  • [MBS-11573] – Collection data missing on ratings pages
  • [MBS-11598] – Broken links generated for deleted artwork missing suffix
  • [MBS-11603] – DNB validation is too strict and blocks legitimate links
  • [MBS-11611] – OverClocked ReMix URLs shouldn’t have a trailing slash after the number.
  • [MBS-11612] – No link to genre/delete from genre index

Improvement

  • [MBS-1588] – More user friendly error when trying to add a CD with a bad TOC
  • [MBS-9251] – Display ended places differently on the map
  • [MBS-9259] – Document how the track list parser works
  • [MBS-9376] – Indicate that edit searches can be bookmarked to save them
  • [MBS-11546] – Allow Resident Advisor /exchange URLs for releases
  • [MBS-11559] – Specify tracklist is empty in Remove medium edits
  • [MBS-11571] – Warn about e-mail as a username
  • [MBS-11587] – Block smart links: hypel.ink
  • [MBS-11590] – Equalize display of empty fields in cover art edits
  • [MBS-11604] – Add genre name to genre index title
  • [MBS-11619] – Ignore periods and +tags in admin e-mail searches

New Feature

  • [MBS-10899] – Report for releases with catalog numbers that look like ISRCs

MusicBrainz Server update, 2021-04-19

Hi! We’re back with another (fairly small) release. Perhaps the most noticeable new thing, other than a few squashed bugs, is that we’re finally officially linking to the 1200px thumbnails of Cover Art Archive images. While these were being generated for new images already for quite a while, we didn’t want to start linking to them until we figured out how to ensure they were also generated for all the images already there. Thankfully, our user (and recent new auto-editor) ROpdebee got in touch with the right people in the Internet Archive and basically made all the efforts needed to ensure this happened. So, we officially have 1200px thumbnails now! That’s some big thumbnails for you.

A new release of MusicBrainz Docker is also available that matches this update of MusicBrainz Server. See the release notes for update instructions.

Thanks to yyoung for improving Bandcamp and DNB URL cleanup. Thanks to CatQuest, chaban, kepstin, Lotheric, navap, selflessself, synthicalist13 and xplt for having reported bugs and suggested improvements. Thanks to ROpdebee for helping us get all the 1200px thumbnails generated. Thanks to Mellthas, mfmeulenbelt, salorock and SistemaRayoXP for updating the translations. And thanks to all others who tested the beta version!

The git tag is v-2021-04-19.

Fixed Bug

  • [MBS-11500] – “Add cover art” edits link to no longer existing release
  • [MBS-11563] – Cache makes newly linked recordings not show in work rels table
  • [MBS-11564] – Niconi Commons “download for free” links blocked by Nicovideo cleanup
  • [MBS-11569] – Collection created from entity sidebar does not restrict entity type
  • [MBS-11581] – Release Group still shows cover art despite Release being darkened
  • [MBS-11582] – Instrument alias sortname guess button breaks page

Improvement

  • [MBS-9631] – Add 1200px to available sizes under cover art images, in release’s Cover Art tab
  • [MBS-9833] – Link relationship style guide within release edit relationships tool
  • [MBS-10534] – Handle both DNB permalinks and DNB search links
  • [MBS-11524] – Disallow *.bandcamp.com/ URLs at release and recording level
  • [MBS-11527] – Block smart links: music.indiefy.net
  • [MBS-11533] – Block smart links: snd.click
  • [MBS-11542] – Update Classical Archives URL format
  • [MBS-11552] – Add validation for Classical Archives links
  • [MBS-11566] – Ensure consistent ordering of appearances sections

Task

  • [MBS-11514] – Move from passing $c to using React.useContext in the sidebar
  • [MBS-11515] – Move from passing $c to using React.useContext in header/footer/menus
  • [MBS-11516] – Move from passing $c to using React.useContext in entity lists
  • [MBS-11517] – Move from passing $c to using React.useContext in MergeHelper

MusicBrainz Server update, 2021-04-05

This time we have a lot of small bug fixes and improvements, and two changes that are likely to be more noticeable. The first one: the list of tags for a user is now paginated, and it shows the upvoted and downvoted tags separately. This is probably good news all around. The second one: long lists of relationships are also paginated. For example, the relationships page for the London Symphony Orchestra now only shows some of the orchestra relationships directly, with a link to “See all 17,303 relationships”. This change was needed, since these pages had become so big for some artists (and areas) that they were timing out since they took more than 30 seconds to load. This reduces the time for the largest pages to a few seconds – still a lot, but much more manageable. We understand that not having all the relationships shown at once might cause some issues with the workflow of some of our hardcore users. Sorry about that, but hopefully it won’t be too bad! If some use cases are very problematic, let us know and we will think about how to make it better.

A new release of MusicBrainz Docker is also available that matches this update of MusicBrainz Server. See the release notes for update instructions.

Thanks to CatQuest, chaban, EndofLine, HibiscusKazeneko, jesus2099, kellnerd, Kid Devine, navap, Sophist and yindesu for having reported bugs and suggested improvements. Thanks to mfmeulenbelt and salo.rock for updating the translations. And thanks to all others who tested the beta version!

The git tag is v-2021-04-05.

Fixed Bug

  • [MBS-8477] – /user/<user>/tag/<tag> pages don’t distinguish between upvotes and downvotes
  • [MBS-8807] – Artist, Label and Series subscribers sub-page wrongly shows “You have no collections!”
  • [MBS-9543] – Seeding a string of “0” results in the field being blanked out
  • [MBS-11320] – When the “I don’t know the tracklist for this medium.” checkbox is unchecked on an existing release, the “Add medium” window pops up
  • [MBS-11321] – Strange things happen when adding the tracklist to a release with exactly 1 unknown medium
  • [MBS-11373] – Relationship merging in relationship editor no longer replaces empty dates
  • [MBS-11386] – Area containment missing from indexed area search results
  • [MBS-11442] – Entering a tag from the Add Cover Art page disables tagging and cover art submission
  • [MBS-11452] – Private user subscriptions tab shown on collection and subscribers page
  • [MBS-11454] – Can’t edit release with format/date clash even if error already existed
  • [MBS-11462] – Private subscriptions/tags/ratings user tabs uselessly shown to admins
  • [MBS-11473] – ISE when loading edit/open while logged out
  • [MBS-11508] – Qobuz favicon is missing
  • [MBS-11510] – Labels for arrows to move tracks up/down are not showing
  • [MBS-11531] – Filtering reports gives ISE
  • [MBS-11538] – Collections data missing in some sidebars

Improvement

  • [MBS-1178] – Relationship view needs paging.
  • [MBS-1870] – Showing a users tags should be paginated
  • [MBS-9310] – Link to created relationship type in Add Relationship Type edits
  • [MBS-9349] – Add Pinterest links to the sidebar
  • [MBS-10691] – Allow seeing tags the user has downvoted in user/tags
  • [MBS-10726] – Several release editor buttons say “disc” but really mean “medium”
  • [MBS-11429] – Don’t block editing if existing URL fails foul of later-added blocking rule
  • [MBS-11432] – Allow also linking places and works to Operabase
  • [MBS-11455] – Make linking of user statistics more consistent
  • [MBS-11467] – Use sort name to order lists of artists for tags
  • [MBS-11476] – Don’t show own open edits in /edit/open
  • [MBS-11485] – Allow irc(s):// links on expand2react
  • [MBS-11503] – Block smart links: trac.co
  • [MBS-11506] – Point home page ‘data’ link to ‘MB Database’

New Feature

  • [MBS-9851] – Add “is not edit author” option when searching for edits by “Edit Note Author”
  • [MBS-11222] – Add a report for releases with media without a tracklist

React Conversion Task

  • [MBS-10690] – Convert user tag list page to React

Our servers are on the move again!

Our servers are on the move again!

You might remember that a few years ago we moved all our servers from the US to Europe. Sadly, that has proven too costly* and prone to fires, so we have decided our servers will continue their travel east and move to India, where hosting them will be significantly cheaper and cooling can be provided by sun-covering swarms of locusts. Most of the non-slacking members of our dev team are based in India by now anyway, so it feels like the right place!

As part of our cost-cutting process, and since air travel is not environmentally friendly anyway, we sent our servers East by boat. Unluckily, that means they had to wait their turn for quite a while in the only traffic jam you can see from space. Luckily that is starting to get resolved now, but it will still take quite a while for the servers to clear the traffic jam, reach their destination and then be located among the twenty thousand containers in the ship carrying them, especially with all other jammed ships arriving at the same time! We’re hoping to be done with it all in a couple months though, after which you can hopefully expect a return to our usual (not-particularly-)fast service.

In the meantime, our whole infrastructure is being hosted on some old servers that our friends at the Universitat Pompeu Fabra in Barcelona donated to us. They claimed they were getting too old for use in a rigorous academic environment, that as we all know in Spain requires systems to work continuously for a few hours every weekday with only a short break for a midday nap. These donated servers are now running from our executive director’s basement. Any issues you have noticed recently are solely caused by this (and have nothing to do at all with the skill and attention span of our development team).

* It could be argued that we should be able to cover the costs given we have had several years of good financial stability by now. While that is true, when we realised that the alternative is to cut costs and spend the savings on fancy ham and beers, the right choice became obvious to us.

Schema change release: May 17, 2021

We’re having a schema change release on May 17, mostly to make small changes that will make our queries more efficient, ensure better constraints, and make some hardcoded options editable without schema changes in the future. We are also upping the required versions of both Perl and Node.js to 5.30 and 16.0 respectively (see the “Minimum version requirements” section below.)

Here’s our list of tickets for the Spring 2021 schema change, with descriptions of what’s being changed:

Schema changes

  • MBS-1424: Add a “first release date” field to recordings. A very popular request for years, this allows requesting the date of the first ever release a recording appeared on. This adds materialized tables recording_first_release_date and release_first_release_date which are updated via triggers whenever the earliest date changes. The change was released as an optional extension to the main MusicBrainz server schema on Dec 16, 2020, but it will be added to the main schema during this schema change.
  • MBS-10208: Allow merging collections. Users who decide two of their collections should be joined into a larger one should be able to do so without having to move all the entities in the collection manually. This requires adding a editor_collection_gid_redirect table (equivalent to other existing x_gid_redirect tables) to ensure the old collection links redirect to the one they have been merged into.
  • MBS-10566: Convert allowed_series_entity_type and allowed_collection_entity_type to tables to allow for additions without schema changes. The constraints allowed_series_entity_type and allowed_collection_entity_type specify which types of entity can be used in series and collections, respectively. As such, if we want to add the possibility to create series of artists, we need to modify the constraint during a schema change. For ease of use, we are moving the constraints to be their own tables instead, allowing us to update them as needed in the future outside of a schema change release.
  • MBS-10647: Add [no label] to b_del_label_special trigger for labels. The b_del_label_special trigger ensures that any attempt to remove a special purpose label fails. Currently it only checks the special case “Deleted label”, but since “[no label]” is also a special purpose label that should never be deleted, we will add its ID to the trigger check.
  • MBS-10821: Remove orphaned recordings from collections for deletion. Replaces a single function, delete_orphaned_recordings(), to add a new clause that makes it so that recordings referenced only in collections (but not linked to anything else in the database) can be deleted as orphans. This was released on the main MusicBrainz servers on June 15, 2020, but it will be added to the main schema during this schema change.
  • MBS-10962 / MBS-11460 – Add materialized tables and indexes to fetch releases and release groups by artist or track artist. These tickets will address performance issues on our current artist pages. They do not modify any existing tables, but as mentioned, add some new tables (to be updated via triggers) and indexes.
  • MBS-11097: Support PKCE (Proof Key for Code Exchange) by OAuth clients. Adds two new columns to the editor_oauth_token table. This feature is opt-in, but allows public OAuth2 clients to mitigate auth code interception attacks. The change was released as an optional extension to the main MusicBrainz server schema on Sep 21, 2020, but it will be added to the main schema during this schema change.
  • MBS-11431: Speed up /ws/js/check_duplicates. Adds new indexes only (on the artist, label, place, and series tables, plus their respective alias tables). Improves some slow queries in the editing interface related to duplicate checking, i.e. finding other entities with the same name. Since this is a non-breaking change, it was released on the main MusicBrainz servers on Mar 15, 2021, but it will be added to the main schema during this schema change.
  • MBS-11451: Support ratings for places. Places can be reviewed in CritiqueBrainz yet cannot be rated in MusicBrainz. This is a strange state of affairs: clearly, if they are worth reviewing in depth, they also deserve the option to rate them. As such, we will add place_rating_raw and place_meta tables, in the same way we have for other ratable entities.
  • MBS-11453: Change entity0_cardinality, entity1_cardinality to SMALLINT. The cardinality columns of the link_type table are used to indicate whether the entity on each side of the relationship is expected to have only a few uses of the relationship type in question, or many (too many to comfortably display/edit). This is what stops every single recording of a work showing up in an edit work page, for example. At the moment we allow only two values for cardinality, 0/1. While it is possible that we will want to allow a few other values in between what is effectively “do not show anywhere ever” and “show all the time”, it is clear we will not need more than 32.000 values. As such, we are moving these columns from INTEGER to SMALLINT to reduce table size.
  • MBS-11456: Add MBIDs and redirect tables for artist credits. Adds a gid column to the artist_credit table, and a new artist_credit_gid_redirect table. The MBIDs will allow public identification of artist credits outside of MusicBrainz, and open the door to useful features in the future.
  • MBS-11457: Drop series ordering_attribute. This column was added back when we were expecting to have different types of ordering attributes for series, but we have never used it. We are planning to just drop the column.
  • MBS-11459: Add a script to create edit_data_type_info. edit_data_type_info is a function used for development that we added in 2020 and we will now add to mirrors as well, both for consistency and for anyone who uses their mirror for development and just wants to use it.
  • MBS-11463: Add view to easily access medium track lengths. Our current way of finding the length of a medium is to either load all its tracks, then sum the durations, or to use the duration of its disc ID when present. The first of these requires a lot more processing than just getting the durations straight from the database, while the second ignores any data tracks not on the CD TOC. This adds a medium_track_durations view that allows easy access to the duration of the track lengths for any medium.
  • MBS-11464: Drop statistics.log_statistic. We added the statistics.log_statistic table for a Google Summer of Code project back during a 2012 schema change, but the work never got finished and implemented. We are not planning to implement it anymore, so this table is useless and we will be dropping it.
  • MBS-11466: Change language.frequency and script.frequency to SMALLINT. The frequency column of a language or script indicates how often it’s used and lets us sort the most frequently used entries at the top of our lists. But we don’t store an exact count, just a number from 0-4 indicating “frequently used,” “hidden,” or “other.” Like MBS-11453 above, we don’t expect to need a full INTEGER type to store these columns at any point in the future, so can safely move them to SMALLINT.

Minimum version requirements

We’re raising the minimum required version for Node.js to v16 (which is the next LTS release coming in April 2021). Our current required version, v10, is hitting the end of its life cycle in April, and given there shouldn’t be a particular difficulty installing the current Node.js LTS on any system, it makes sense to just upgrade to the most recent LTS by the time of the schema change day.

We’re also raising the minimum required version for Perl to 5.30. The latest version is 5.32, but the current Ubuntu LTS (20.04), which is likely to be the next base image for our Docker containers, only provides 5.30. Our current required version, 5.18, was released all the way back in 2013, so moving to 5.30 is already a fairly significant improvement.

We’ll post final details about this release just prior to the release and shortly after we complete it, including instructions on how you can update your own copy. If you have any questions, please do leave a comment below or on the linked JIRA tickets!