MusicBrainz Server update, 2019-06-03

This server release mostly fixes bugs and regressions. It also updates the URL cleanup code and continues the conversion of templates to React.

Thanks to CatQuest, cyberskull, hibiscuskazeneko, Jeluang, Lotheric, mfmeulenbelt, tungolcraft, yeeeargh, and everyone who tested the beta version, reported issues, or updated website translations.

The git tag is v-2019-06-03.

Bug

  • [MBS-8826] – /ws/ requests that 404 should not return HTML
  • [MBS-10141] – Labels from Trois-Rivières should appear 1 layer up in labels from Mauricie
  • [MBS-10147] – Reports break if an entity in them has been removed / merged
  • [MBS-10148] – Can’t change entity type of empty collection
  • [MBS-10153] – Database statistic tables’ headers’ icons are 404s
  • [MBS-10160] – YouTube channel link that uses the https://www.youtube.com/c/username format won’t have the username shown in the sidebar
  • [MBS-10163] – “Javanese” does not show in instrument Alias page
  • [MBS-10167] – “Existing medium” results give medium position, not number
  • [MBS-10171] – ISE when calling ws/js with a wrong ID
  • [MBS-10182] – Some artist /relationships tabs show an internal server error
  • [MBS-10183] – Regression: Display bug in the title of page crediting the artist (Release Group, Release, Recording)
  • [MBS-10184] – Regression: ISE on displaying some edits
  • [MBS-10196] – XSS vulnerability in Knockout for IE <= 7

Improvement

  • [MBS-9762] – Standardize Songkick URLs
  • [MBS-10095] – Provide a better error than an ISE if trying to admin edit non-existing user
  • [MBS-10108] – Allow linking individual tweets to recordings
  • [MBS-10142] – Update URL cleanup for Sina Weibo URLs + add them to the sidebar
  • [MBS-10151] – Update LiveFans URLs to use HTTPS
  • [MBS-10152] – Display artist areas on Artists tab of area entity

React Conversion Task

  • [MBS-9849] – Convert the release group public pages to React
  • [MBS-10104] – Convert the area index page to React
  • [MBS-10127] – Convert root/components/relationships-table.tt to React
  • [MBS-10189] – Convert the place performances page to React

Google donates $10,000 in cloud computing credits. Thank you!

The Google Open Source Programs Office continues to support MetaBrainz in a number of ways and most recently they donated $10,000 in credit toward their cloud services. Thank you Google!

This credit allows us to run some services in the cloud to round out primary hosting setup — this gives us a some redundancy and allows us to not keep all of our critical eggs in one basket. We can also give our open source developers Virtual Machines from time to time, since a lot of our projects are very data heavy. Having access to a fat VM can sometimes turn a really frustrating project that makes your laptop melt into a project that is satisfying to watch chug along.

Thank you again, Google, the Open Source Programs Office and in particular, Cat Allman!

MusicBrainz schema change release, 2019-05-13 (with upgrade instructions)

We’re happy to announce the release of our May 2019 schema change today! Thanks to all who were patient during today’s downtime as we released everything to our production servers.

This is a fairly minor release as far as schema changes go, but please do report any issues that you come across, especially any related to genres and collections.

Visible changes with this release are limited to an indication if a specific artist credit is being edited (MBS-5387). Work on some of the changes to collections and genres is quite advanced, and we’re hoping to release some of the new features onto beta already in a week or so from now, while others might take a while longer.

Now, on to the instructions.

Schema Change Upgrade Instructions

Note: Importing the latest data dump is always a valid alternative to running ./upgrade.sh on an existing database, if you’d prefer to also get new data in one go. Just follow the relevant instructions in INSTALL.md. The git tag is v-2019-05-13-schema-change. The rest of the instructions here assume an in-place upgrade.

  1. Make sure DB_SCHEMA_SEQUENCE is set to 24 in lib/DBDefs.pm.
  2. If you’re using the live data feed (your REPLICATION_TYPE is set to RT_SLAVE), ensure you’ve replicated up to the most recent replication packet available with the old schema. If you’re not sure, run ./admin/replication/LoadReplicationChanges and see what it tells you; if you’re ready to upgrade, it should say “This replication packet matches schema sequence #25, but the database is currently at #24.”
  3. Take down the web server running MusicBrainz, if you’re running a web server.
  4. Turn off cron jobs if you’re automatically updating the database via cron jobs.
  5. Switch to the new code with git fetch origin followed by git checkout v-2019-05-13-schema-change.
  6. Install newer dependencies Yarn and NodeJS 8 or later according to install prerequisites.
  7. Run cpanm --installdeps --notest . (note the dot at the end) to ensure your perl-based dependencies are up to date.
  8. Run ./upgrade.sh (it may take a while to vacuum at the end).
  9. Set DB_SCHEMA_SEQUENCE to 25 in lib/DBDefs.pm as instructed by the output of ./upgrade.sh.
  10. Turn cron jobs back on, if applicable.
  11. Restart the MusicBrainz web server, if applicable. It’s also recommended you restart redis. If you’re accessing your MusicBrainz server in a web browser, run ./script/compile_resources.sh.

Here’s the list of resolved tickets:

Bug

  • [MBS-5387] – ACs being edited aren’t marked as having pending edits on the aliases tab
  • [MBS-9365] – event_meta_fk_id was never created as part of any upgrade script
  • [MBS-9462] – Standalone databases created before schema 21 are missing some l_event_url triggers
  • [MBS-10146] – Regression: ISE on Remove DiscID page
  • [MBS-10149] – Swap track titles with artist credits fails to update both fields properly
  • [MBS-10150] – Regression: The link to the release group reviews in the release page is broken

Improvement

  • [MBS-9664] – Add database constraints to disallow loop relationship
  • [MBS-10044] – Add place area to place lists

Database Schema Change Task

  • [MBS-10052] – Add new schema for the event art archive
  • [MBS-10173] – Create a genre table in the DB and populate it with existing genres
  • [MBS-10174] – Create an addition timestamp in the DB for new collection items
  • [MBS-10175] – Create a position integer in the DB for collection items
  • [MBS-10176] – Create a comment text field in the DB for collection items
  • [MBS-10177] – Create an editor_collection_collaborator table for collaborative collections
  • [MBS-10178] – Create a genre_alias table
  • [MBS-10181] – Create filesize for cover art and each thumb in the DB

React Conversion Task

  • [MBS-9925] – Convert collection pages to React
  • [MBS-10179] – Convert all entity list components to React

MusicBrainz Schema change upgrade downtime: 17:00 UTC (10am PST, 1pm EST, 19:00CEST)

Hi!

At 17:00 UTC (10am PST, 1pm EST, 19:00CEST) we will start the process of our schema change release. The exact time that we plan to start the change will depend on how long it takes to finish our preparations, but we expect it to be shortly after 17:00UTC.

Once we start the process we will put a banner notification on musicbrainz.org and we will also post updates to the @MusicBrainz twitter account, so follow us there for more details.

After the release is complete, we will post instructions here on how to upgrade your replicated MusicBrainz instances.

Google Summer of Code 2019: Accepted students and their projects

The accepted students for Google Summer of Code have just been announced! We’re please to announce that Akhilesh Kumar (BookBrainz), Aidan Lawford-Wickham (AcousticBrainz), Vansika Pareek (ListenBrainz), Anirudh Jain (MusicBrainz), amCap1712 (MusicBrainz) and Shamroy Pellew (CritiqueBrainz) have been accepted on behalf of the MetaBrainz Foundation!

To find out more about the accepted students and what they will be working on, please take a look at the list of accepted projects.

This year was quite challenging to decide which students to accept. We had more good proposals than we could accept — which is quite heartbreaking, since we hate having to turn away good proposals. Still, we have a very good spread of students across our projects and we’re quite excited for Summer of Code this year.

Thanks to everyone who applied, all of our mentors and of course, Google’s Open Source Programs Office for making Summer of Code a reality.

MusicBrainz Server update, 2019-04-26

We finally have a new release! Some large conversions of the MusicBrainz website templates to React happened during the past three months, leading to a long delay between releases – sorry about that! This release also fixes several bugs and adds some small improvements.

Thanks to Cyna, Ge0rg3, and spellew for their contributed code, mostly written during GCI. Also, thanks to alpinetux, amCap1712, chirlu, culinko, cyberskull, derobert, Griomo, hibiscuskazeneko, Jeluang, jessew, kepstin, Kid Devine, ListMyCDs.com, Lotheric, mfmeulenbelt, rafwuk, RavenWorks, samj1912, yindesu, yurim, and everyone who tested beta version, reported issues, or updated website translations.

The git tag is v-2019-04-26.

Bug

  • [MBS-6895] – Cannot inc=tags when browsing releases
  • [MBS-8634] – Privileged user accounts page still mentions the style council
  • [MBS-9013] – Removing specific ordering of relationships doesn’t create an edit
  • [MBS-9276] – Transient test failure in Edit::Instrument::Merge
  • [MBS-9297] – “Set track lengths” from Disc ID with no change should not be allowed
  • [MBS-9309] – Mediums fail to merge when two recording positions are swapped
  • [MBS-9400] – Tags are missing from artist nodes in /ws/2/release JSON requests
  • [MBS-9413] – Older “Remove relationship” edits can’t be loaded
  • [MBS-9669] – UI language menu not available in Spanish and Greek
  • [MBS-9845] – country.null stats not being collected
  • [MBS-9893] – Batch voting check boxes disappeared
  • [MBS-9899] – Recording and recording artist tags/genres missing in JSON WS
  • [MBS-9943] – Autocomplete.js is pretty much untranslatable
  • [MBS-9975] – Clicking the “A-a” button (guess sort name) to add a release alias sort name does not work.
  • [MBS-9983] – Non-group artists statistics does not include characters
  • [MBS-9985] – “Last updated on Invalid date” on release sidebar
  • [MBS-9986] – RG types not translated on sidebar
  • [MBS-9991] – Relationship names are not translatable
  • [MBS-9993] – MusicBrainz server should allow CORS requests to /oauth/token
  • [MBS-9995] – Space missing in autoeditor election text
  • [MBS-10009] – Some sidebar types are not translated
  • [MBS-10021] – ArtistRoles is not translatable
  • [MBS-10027] – t::MusicBrainz::Server::Controller::Admin::EditBanner fails randomly
  • [MBS-10031] – Tooltip says ‘null’ for artist names in ‘writer’ column of ‘work’ search results
  • [MBS-10033] – Missing space after Wikipedia blurb
  • [MBS-10045] – Regression: External links to Anison Generation in the sidebar appends https
  • [MBS-10075] – Release with no mediums disappears from release collection when sorting on format/no. of tracks
  • [MBS-10081] – Invalid last updated date on search result pages
  • [MBS-10084] – Unable to add CDBaby URL containing “ï”
  • [MBS-10085] – Long URL fails to wrap on release page in Chrome
  • [MBS-10088] – Example search for relationships breaks with cardinality
  • [MBS-10090] – Loading a doc page containing %3F loads full wikipage html
  • [MBS-10099] – Lyrics URL relationship uses the wrong link phrase on work edit page

Improvement

  • [MBS-6574] – Recursive area tabs
  • [MBS-7463] – Add redirect from /tag to /tags
  • [MBS-8782] – Show artist name on “Edit Note” tab of “Edit Release” page
  • [MBS-8922] – User reporting: Remove “Reveal my email address” option (or default it to on)
  • [MBS-9340] – Don’t allow more languages if [No lyrics] is selected
  • [MBS-9656] – [Change release quality] → [Change data quality]
  • [MBS-9935] – Indicate what to do with annotation reports
  • [MBS-9936] – Move from CJS style to ES6 (imports, exports)
  • [MBS-9988] – On /tags page, allow to go back to not showing downvoted tags
  • [MBS-9989] – Add new “Not applicable” gender option to stats
  • [MBS-9990] – Make formatUserDate error rather than return wrong date if given object
  • [MBS-9998] – Improve cleanup of Facebook URLs
  • [MBS-10008] – Generalize and localize CPDL link’s title in the sidebar
  • [MBS-10010] – Simplify sidebar_name methods for URLs
  • [MBS-10029] – When viewing edits by editor X, include editor X in the page title
  • [MBS-10032] – Detect and clean iTunes “author” URLs for artists
  • [MBS-10063] – “No linguistic content” → “[No lyrics]” in work edits display
  • [MBS-10074] – Display – on release search results for format/tracks if no medium exists
  • [MBS-10094] – Right-justify barcode on release group page
  • [MBS-10101] – Allow SecondHandSongs performance URLs for Recordings
  • [MBS-10102] – Use the containing country area of artists and labels for country statistics
  • [MBS-10103] – Update Bandsintown URL cleanup to strip languages

New Feature

  • [MBS-10011] – Congratulate/thank users on their birthdays
  • [MBS-10022] – Event tab for areas

React conversion task

  • [MBS-9632] – Convert statistics to React
  • [MBS-9699] – Convert the user account pages to React
  • [MBS-9904] – Convert the list of privileged users to React
  • [MBS-9909] – Convert report templates to React
  • [MBS-9926] – Convert entity subscribers pages to React
  • [MBS-10005] – Convert “Details” and ”Tags” tabs, for each entity, to React
  • [MBS-10007] – Convert “Add annotation” edit display to React
  • [MBS-10014] – Convert “Add area” edit display to React
  • [MBS-10015] – Convert “Add artist” edit display to React
  • [MBS-10016] – Convert “Add event” edit display to React
  • [MBS-10017] – Convert artist/cannot_split.tt to React
  • [MBS-10018] – Convert artist/special_purpose.tt to React
  • [MBS-10019] – Convert the artist and place event pages to React
  • [MBS-10026] – Convert the Labels page for Area to React
  • [MBS-10042] – Convert “Aliases” tab, for each entity, to React
  • [MBS-10046] – Convert /admin/banner/edit to React
  • [MBS-10049] – Convert user/report to React
  • [MBS-10086] – Convert attributes admin lists to React
  • [MBS-10091] – Convert root/components/relationships.tt to React

Other task

  • [MBS-8360] – Show BookBrainz relationships in the sidebar
  • [MBS-9082] – Standardize SecondHandSongs URLs
  • [MBS-9932] – Remove or update “Releases with superfluous data tracks” report
  • [MBS-9999] – Allow setting custom redirect URI while registering applications to be installed
  • [MBS-10035] – Add text to the “Report editor” page asking for links
  • [MBS-10036] – Add ccMixter.org to other database whitelist
  • [MBS-10037] – Send editor reports to MB user admin mailing list
  • [MBS-10080] – Relabel “MusicBrainz Blog” link as “MetaBrainz Blog” on MusicBrainz frontpage

Mini ListenBrainz update released today

Following up on our release from last week, we found a number of minor problems in production that were really hard to spot on our test setup. Sometimes you need to have real data flowing through your system before you can find the real problems.

The following pull requests were merged and released just now:

This should hopefully make the follow page work a little better for everyone. 🙂