Category: Server

Search server update for 2013-10-14 release

In conjunction with our main server release we’ve released a new search server. Thanks for your hard work on this release, Paul!

This release contains changes to match the Autumn 2013 database schema changes plus some bug fixes and improvements mostly to do with artist credits and aliases. See the detailed list of issues that are fixed in this version:

Bug

  • [SEARCH-46] – Searching for a release by releasename and artists english name will give no matches
  • [SEARCH-122] – Recording Search only adds release artist credit if different to recording artist credit and VA release
  • [SEARCH-304] – Webservice Json output for aliases when searching is inconsistent for boolean values
  • [SEARCH-311] – Recording, release and releasegroup search results incomplete artist aliases
  • [SEARCH-324] – Release search should return catalog numbers in ascending order

Improvement

  • [SEARCH-132] – Allow to filter release search by quality
  • [SEARCH-242] – The searcher can’t guess that the artist name one’s looking for is alias or artist name ☞ look for both
  • [SEARCH-288] – Include alias sortnames when searching works

Task

2013-10-14 schema change release update instructions

As promised, here are the instructions for updating any instances of MusicBrainz you might have. You will need to perform these steps to upgrade to the new version:

  1. Take down the web server running MusicBrainz, if you’re running a web server.
  2. Turn off cron jobs if you are automatically updating the database via cron jobs.
  3. Make sure your REPLICATION_TYPE setting is RT_SLAVE
  4. Switch to the new code with git fetch origin followed by git checkout v-2013-10-14
  5. Run carton exec -Ilib — ./upgrade.sh (or simply ./upgrade.sh if you aren’t using carton, such as for VM users or fairly new installs).
  6. Set DB_SCHEMA_SEQUENCE to 19 in lib/DBDefs.pm
  7. Turn cron jobs back on, if needed.
  8. Restart the MusicBrainz web server, if needed.

If your server fails to start or cron is having issues, it may be a missing perl module problem. To check for any missing perl modules, follow the instructions in INSTALL.md.

October 14 schema change complete

We’ve just finished rolling out the fall schema change release. Apart from the /place/create endpoint failing, the site is back up in full read-write mode now.

Read on for the changes in this release — our next blog post will give instructions on how to upgrade your instances of MusicBrainz.

Thanks to everyone who worked on this release!

Bug

  • [MBS-2301] – Attach TOC to new release – TOC/DiscId is lost when based on existing release/tracklist
  • [MBS-4453] – Duplicate artist credits
  • [MBS-5624] – Release groups don’t show the yellow removal warning when empty
  • [MBS-5647] – Release header of a different release appears on the release duplicates tab of the release editor
  • [MBS-6067] – Internal server error when using query parameter with a lookup in the webservice
  • [MBS-6211] – Aliases are missing the “ended” flag
  • [MBS-6373] – Area names being incorrectly translated
  • [MBS-6518] – Wikidocs pages keep 404ing
  • [MBS-6703] – Type displays as (none) on remove cover art edits.
  • [MBS-6715] – Internal server error looking up non-existing ISWC
  • [MBS-6717] – Some false reports of possible Artist collaborations
  • [MBS-6736] – ISE when giving a non-integer to entity/mbid/annotation/
  • [MBS-6748] – Internal server error loading some edit relationship edits
  • [MBS-6765] – Documentation search doesn’t work over https in recent Firefox/Chrome
  • [MBS-6782] – Artist overview page still displaying release date that was removed
  • [MBS-6787] – edit search merge works edits : ISWC are duplicated

Improvement

  • [MBS-631] – Add support for deprecating a relationship
  • [MBS-6068] – Remove the _name tables
  • [MBS-6069] – Track MBID webservice changes
  • [MBS-6182] – Deleted editors should be marked in a real way, not just designated by lack of password/well-known username
  • [MBS-6392] – Display ISNIs with spaces
  • [MBS-6543] – Highlight specified edit note when using edit note fragment in URL
  • [MBS-6564] – Add disambiguation comments to areas
  • [MBS-6706] – Improve the relationship type documentation display
  • [MBS-6713] – Ensure IMDB links are added at the right level
  • [MBS-6767] – Remove the Creative Commons download relationships report
  • [MBS-6779] – INSTALL.md doesn’t mention that you need to apt-get install cpanminus before trying to run cpanm.

New Feature

  • [MBS-5701] – Add a way to mark recordings as containing video
  • [MBS-6200] – Add a “place” entity
  • [MBS-6683] – Add autoselect for ReverbNation URLs

Task

  • [MBS-6046] – Remove PUID support
  • [MBS-6669] – Update the Allmusic logo used in the sidebar
  • [MBS-6732] – Add neyzen.com to score whitelist
  • [MBS-6766] – Delete unused root/release/full.tt

You can check out this release with the following tag: v-2013-10-14 .

Schema change release tomorrow at 1700UTC

Tomorrow, Monday 14 October, 2013, 17:00 UTC (10:00 PDT, 13:00 EDT, 18:00 BST, 19:00 CEST) we’re going to release our next round of schema changes!

As it is typical with our fall schema changes, this one is a little simpler and more focused on cleanup, rather than massive new changes. This gives me hope that we will have smoother release than we did in the spring. 🙂

We’re going to make the site read-only and run off our backup database server while we upgrade our primary database server. I suspect that we should be in read-only mode for about an hour. The exact start time is not quite known — we’ll start our release process at 17:00 UTC, but when we go to read-only is hard to tell. We’ll tweet and give a shout in IRC when we’re ready.

Server Update 2013-09-30

With the MusicBrainz summit happening just this weekend, and most work continuing on the upcoming schema change release – this server update is a tad on the light side! However, we do at least have something to show. Thanks to nikki for the main change in this release:

Bug

  • [MBS-6735] – ProcessReplicationChanges breaks with update packages that cross the limit boundary

New Feature

  • [MBS-5789] – Make links for HTTPS enabled sites use schema independent links

The Git tag for this release is v-2013-09-30.

Server Update 2013-09-16

Hello all. We’ve just pushed out another small little update to MusicBrainz. This release is small while we continue to focus on getting the upcoming schema change ready, so there are a handful of bug fixes and improvements. Thanks to Frederik “Freso” S. Olesen, Michael Wiencek and the rest of the MusicBrainz team for their work on this release. Here’s what’s changed:

Bug

  • [MBS-2521] – Setting fields via query parameters blanks all default values (including auto-editor checkbox)
  • [MBS-6163] – Following INSTALL does not result in a runnable server
  • [MBS-6279] – /ws/2/discid/ ignores DB_READ_ONLY
  • [MBS-6281] – Logged in users can load /register
  • [MBS-6287] – Adding disc IDs ignores DB_READ_ONLY
  • [MBS-6291] – Adding releases to your collection via the webservice ignores DB_READ_ONLY
  • [MBS-6607] – Internal server error after logging in from search results page
  • [MBS-6689] – Pasting an MBID initiates a search
  • [MBS-6691] – Inline search for works should include localised name
  • [MBS-6695] – URL cleanup doesn’t catch/clean iTunes URLs containing +
  • [MBS-6700] – Internal server error displaying a relationship
  • [MBS-6707] – Internal server error trying to authenticate via the webservice if password needs resetting
  • [MBS-6716] – Tagger button doesn’t work when using https in some browsers
  • [MBS-6721] – beta: Internal server error in /ws/js/work
  • [MBS-6743] – URL cleanup doesn’t catch/clean iTunes URLs containing *

Improvement

  • [MBS-6419] – Don’t check any artist credits by default when editing artist names.

Task

  • [MBS-6702] – Add Rockipedia to the Other Databases whitelist
  • [MBS-6708] – Turn ISNI number into link

The Git tag for this release is v-2013-09-16.

Changes for upcoming schema change release: 2013-10-14

For our fall schema change release we’re going to fix the issues listed in this Schema change, 2013-10-14 fix version. This schema change, much like our previous fall schema changes, is going to be a little lighter than our spring changes. The two big changes that you should be aware of are:

  • MBS-6046: Remove PUID support. PUIDs have been deprecated for quite some time in favor of using AcoustID. This change completes our switchover to AcoustID.
  • MBS-6068: Remove _name tables. With NGS we introduced tables that isolated all of the name strings for a given table, in order to reduce duplicate strings in our database. Sadly, this proved to be more pain that gain, so we’re going to remove them. This will make writing queries for our schema much easier. Sadly, this means that if you use the MusicBrainz database directly, (as opposed to using our Web Service, which we recommend) then you will likely need to update your SQL queries.

The rest of the changes will have a much smaller impact: Adding a disambiguation column for areas, adding places, and adding the ability to mark a relationship as deprecated. We’re also making some minor changes to our non-replicated tables that will not be visible to our Live Data Feed users, but are more convenient to make during a schema change.

The scheduled date for this release is Monday, 14th October, 2013.

Our RDFa dilemma

A few years ago Queen Mary University was awarded a grant to implement modern RDF support in MusicBrainz. The RDFa portion was implemented on our server and has been in our pages for quite some time.

However, the code to implement RDFa is brittle and has not been maintained through a number of schema changes and is quite broken at this point in time. When wondering if we should fix this or remove it, we could find no one or no application that we know of, that makes use of the embedded RDFa in our pages. And no one stepped up to fix it and the author of this code is not responding to emails inquiring about this.

At this point, we’re ready to remove the broken code from our pages in an effort to remove technical debt that has accumulated over the past few years. If you care about RDFa support in our pages, please speak up now. Ideally anyone speaking up would also volunteer to adopt the RDFa code and see it through life as our schema changes.

We’re going to take the HTTPS plunge!

Yesterday in our dev meeting we agreed to take the HTTPS plunge for all of our web site traffic in as little as 2 weeks time. This means that all web site traffic (not the web service) will be served over HTTPS; if you visit any MusicBrainz HTTP URL (e.g. http://musicbrainz.org ) you will be redirected to the equivalent HTTPS URL (e.g. https://musicbrainz.org ). This will not be applied to our web services, you’ll still be able to access those with HTTP. However, we do encourage all of our web service users to make use of HTTPS when possible.

We have one bug to address before we make this switch. And if we can find a sufficient fix for this in time, we’re going to make the HTTPS switch on 16 September 2013. If we can’t find an acceptable fix, we’ll have to postpone this switchover.

If for some reason you can see that switching all web site traffic to HTTPS is a bad idea, please leave us a comment ASAP.

Server Update 2013-09-02

We have a tiny release to kick off September, as we’ve been focusing quite heavily on tightening up the reliability and ease-of-maintenance of the servers themselves. However, we still have a few important bug fixes to release. Here’s what’s changed:

Bug

  • [MBS-4730] – Sort names for artist search hint aliases are displayed, but cannot be edited.
  • [MBS-6218] – When a Person with a gender is merged into a Group, ModBot ISEs.
  • [MBS-6561] – End column empty in search results for ended artists with no end date
  • [MBS-6635] – Several reports not updated since May 15
  • [MBS-6690] – JSON web service ISE with inc=aliases
  • [MBS-6698] – Wiki excerpts truncated by mid-sentence initials (eg, of names) that are mistaken for periods ending a sentence.

The Git tag for this release is v-2013-09-02