Server capacity update

Zas and I have been working hard to improve the capacity and stability of the site. In the last week, we’ve identified and fixed at least 3 problems with the search servers and we’ve added a timeout function that times out queries that take longer than 3 seconds. We think that the main cause of trouble was that queries were piling up after a slow query ran too long and that the servers never recovered from that and consequently crashed.

We won’t go as far as saying that the search servers are fixed — every time we have a smidgen of hope that things are improving, they crash again. Seemingly out of spite! So, the search servers are better. 😉

Zas has also made a number of changes to the gateways and how we rate limit our incoming traffic. The rate limiting is now being done in a smarter way that reduces the overall traffic on our web servers. Well done!

We’ve also increased our bandwidth budget by 4mbits per second, which makes the site feel considerably more responsive.

Let me put these improvement into numbers: About a week ago were were struggling to keep up 250 requests per second and the site felt very sluggish. Now we can handle 500 requests a second and the site feels considerably faster. For large chunks of the day we are managing to handle all the traffic we should handle. And, the search servers haven’t crashed in 4 days!

We hope that this will give us a solid base from which to release the scheme upgrade tomorrow. Then once that is complete, we will start work on moving to the new hosting company.

Thanks for being patient with us!

Important: Schema change delayed to May 23

With our ongoing hosting issues due to massive traffic increases and failing hardware we’ve been too distracted trying to manage those issues to finish all of the testing for the schema change release that was scheduled for today.

We deeply regret having to do this, but we’re going to delay the schema change release by a week. It is now scheduled for May 23, 2016. This week long delay will give us a chance to further tweak our server configuration (more on this in the next blog post) and to test the schema change release in much more detail.

We are, however, going to upgrade our database server to Postgres 9.5 either later today or tomorrow. During this upgrade we are going to employ a back-up database server and keep MusicBrainz running in read-only mode with a slightly reduced overall capacity (I’m sure everyone know what that means by now). This upgrade should have no other effects on our downstream data users.

We will give people plenty of notice before we start the postgres upgrade via our site banner and via our Twitter account (@musicbrainz).

Sorry for the continued drama affecting our services — we’re working hard to keep things together!

Server update, 2016-04-04

This is a small bug-fix release while we work on finishing the May schema change update. Thanks go to reosarevok and ethus3h for their patches this time around. The git tag is v-2016-04-04 and you can find the complete changelog below.

Bug

[MBS-8850] – No events tab for tags
[MBS-8861] – Vertical spacing off on editor profile if “last login” is missing (account admins only)
[MBS-8874] – Editing an entity sometimes shows it as a possible duplicate of itself
[MBS-8886] – Header menus should work without JavaScript

Improvement

[MBS-8591] – Increase pagination item count

Server update, 2016-03-21

The most noticeable change in this release is that we’re using Wikidata links to fetch images for any entity that has them. Thanks to Roman Tsukanov for working on this cool new feature! We’ve also made the header menus require clicks to open and close, and fixed several bugs listed below. The git tag is v-2016-03-21.

Bug

[MBS-7914] – alias list not included in track level artist credits when fetching release information
[MBS-8837] – Event edits fail to apply with ERROR: invalid input syntax for type time: “1970-01-01T19:00:00”
[MBS-8848] – Own private collections ignored on entity pages
[MBS-8858] – Edit medium edit stuck trying to use a deleted recording
[MBS-8862] – Search edits: release group condition is broken

Improvement

[MBS-6381] – Use Wikidata URLs to fetch images for entities
[MBS-8843] – Require clicking on the header menus to open them

New Feature

[MBS-6152] – Provide a better way to list private collections in /ws/2/release output

Task

[MBS-8805] – Drop the completely unused pre-NGS tables on totoro

May 2016 schema change release details

In about two months time we’ll have the next schema change release: May 16, 2016. Even after skipping the fall schema change release, this release is going to have few changes that will impact our downstream users. Most of the tickets in this release will make minor improvements to database indexes and edit tables. If you are one of the few users of our edit data, then you should delve deeper into the list of tickets in this release. For everyone else, I will summarize the tickets with a greater impact.

In a previous blog post we also talked about upgrading the minimum required version of postgres. We received no real feedback requesting for us to upgrade to 9.4, but we did receive some feedback that some people would prefer 9.5, which is our preference as well. Based on that feedback, we’re going to make PostgreSQL version 9.5 the minimum required version. If you’d like to run a MusicBrainz replicated instance via our Live Data Feed, you will need to run Postgres 9.5!

The official minimum supported Ubuntu release as of now is still Ubuntu 10.04 LTS (Lucid Lynx) which reached end-of-life a year ago. We will upgrade that to Ubuntu 14.04 LTS (Trusty Tahr) at the schema change release. In particular, this means that we might start using Perl 5.18 features in the MusicBrainz Server code (as opposed to Perl 5.10 currently).

We understand that this is potentially a lot of work for some of our users, but occasionally we need to upgrade our requirements. We try and limit these sorts of upgrades as much as possible, so please bear with us.

Finally onward to the details of the release. Please take a look at the list of issues that will be addressed in this release. The few tickets worth discussing in details are:

MBS-8838 – “Add gids to all *_type tables“. This ticket adds MBIDs (GIDs in schema lingo) to all of our tables that define a type for some database element. Given that we recommend that external users never reference our data by row ids, we really need to provide proper permanent MBIDs to all elements of our database.
MBS-6024 – “Support more than one barcode on same release (SQL edition)“. This ticket adds the ability for the database to contain more than one barcode for a given release. However, this ticket does not include the user interface portions of this feature. The team will add the user interface/edit portions of this feature in a later, non schema change release.
MBS-4501 – “Alternative tracklists“. This ticket creates a new feature that would allow an alternative tracklist to be used for a given release. This is a better solution for handling conflicts between our style guidelines and how the data appears on the release. It is also a more elegant solution for translations of releases into different languages.

As usual, we will post final details about the release shortly before the release happens. If you have any questions about this release, feel free to ask specific questions in the tickets or general questions in the comments below.

(Edited 2016-03-16 at 12:55 UTC to add the upgraded Ubuntu requirement.)

Server update, 2016-03-07

This release fixes the web service to support all the new entity types that can be added to collections since the last release, including to support submission, and introduces a few new browse requests (browsing collections by entity gid or editor name; browsing entities by collection gid). These are documented at https://musicbrainz.org/doc/Development/XML_Web_Service/Version_2.

Area pages now have a “Users” tab where you can see other editors from that area, provided they’ve filled out that info in their profile.

A “single sign-on” endpoint has been added for logging into our new community forum, to replace the previous OAuth2 method. The main benefit over OAuth2 is that it now keeps peoples’ usernames and email addresses in sync with their MusicBrainz account.

Thanks to Roman Tsukanov (Gentlecat) and Frederik “Freso” S. Olesen for their work on today’s release. The git tag is v-2016-03-07 and the complete changelog is below.

Bug

[MBS-3125] – collection queries via the webservice are broken
[MBS-5323] – GET request for releases in a collection returns unordered results
[MBS-8459] – WS doesn’t have work collection endpoints
[MBS-8651] – collection add and delete endpoints
[MBS-8652] – PUT requests for invalid webservice collection URLs return the same content as a GET
[MBS-8825] – Links in the header force the default cursor via CSS

Improvement

[MBS-6120] – web service browse release where collection equals mbid

New Feature

[MBS-6152] – Provide a better way to list private collections in /ws/2/release output
[MBS-6511] – Show tab for “editors” on an area page when logged in
[MBS-8839] – Implement Discourse SSO endpoint

Task

[MBS-8841] – Update links to forums
[MBS-8842] – Update Picard’s logo on https://musicbrainz.org/

Server update, 2016-02-22

New Style

The style of the website has changed to use our new logo and better match the rest of the *Brainz family. It may be rough around the edges in some areas, but we’ll continue to make improvements as we receive feedback. Thanks to Roman Tsukanov (Gentlecat) who did all of the initial work on this.

Other Changes

Nicolás Tamargo (reosarevok) has fixed our header menus to be clickable on touch devices, and we have some other fixes and improvements from Ulrich Klauer (chirlu) and myself listed below. The git tag for today’s release is v-2016-02-22.

Bug

[MBS-8007] – Can’t change collection type of existing collection
[MBS-8624] – Components for server-side rendering are not equivalent to TT macros
[MBS-8771] – Contact-URL on start page result in “Page Not Found”
[MBS-8772] – Big Audio Dynamite is high-lighted, but no open edits seem around
[MBS-8776] – JSON-formatted collection release list shows page total and not overall collection total
[MBS-8789] – “{user} has not tagged anything” in user/tags seems broken
[MBS-8801] – “A group of artists cannot have a gender” when removing the gender

Improvement

[MBS-739] – Menu items with dropdowns should not have actions under them (other than the drop down opening)
[MBS-6212] – No link to edit profile on the profile
[MBS-8004] – Extend collections to other entities
[MBS-8787] – Add CORS headers to JSON-LD responses
[MBS-8790] – “ended” shows differently in add and edit relationship edits

Task

[MBS-8778] – Point About → Team to meb.o/team
[MBS-8779] – Restructure footer and add MetaBrainz to it
[MBS-8784] – Add Triple J Unearthed to the Other DBs whitelist

Upgrading Postgres for MusicBrainz Live Data Feed users

We’re slowly approaching that time of year: Schema change release time. After skipping our fall update to focus on some internal tasks, we’re ready to have another schema change release in the spring: May 16, 2016

We have started the process to collect features we wish to release for this schema change release and we’ll be publishing that list in the coming weeks. However, we’re contemplating the impact of one more change we’d like to make: Upgrading to a more recent version of Postgres.

Internally we are going upgrade to Postgres 9.5, which was recently released, so we expect that the Postgres team will have worked out the most significant kinks before we’re ready to move to it. However, even though we are moving to 9.5, we are considering the impact on our downstream users/customers who need to make the same or similar change.

While we are moving to version 9.5 of Postgres, we have the option of only adopting features from Postgres 9.4, which means that our downstream users may continue to use Postgres 9.4. However, Postgres 9.5 has some nice features we’d like to use (e.g. UPSERT), so we’re pondering if it is possible for us to require Postgres 9.5 from all of ours Live Data Feed users starting on May 16, 2016.

We have already informally queried a few of ours users and so far it seems that requiring Postgres 9.5 is feasible. If you are a Live Data Feed user and feel that this requirement of Postgres 9.5 is too much for your and your organization by May 16, 2016, please leave a comment to this blog post!

Server update, 2016-02-08

This update hopefully fixes some issues with “Edit Medium” edits that, in rare cases, resulted in an incorrect track listing. Sometimes tracks were being inexplicably deleted. The git tag for today’s release is v-2016-02-08.

Bug

[MBS-8752] – Database inconsistencies when updating medium
[MBS-8765] – instrument_annotation should not be backed up in mbdump.tar.bz2
[MBS-8770] – Banner not displayed

Task

[MBS-7475] – Get rid of Algorithm::Merge

Server update, 2016-01-25

Backwards-Incompatible JSON Web Service Changes

In an effort to get our JSON Web Service out of “beta” status, we’ve made some backwards-incompatible changes to it in this release:

The video flag on recordings is now outputted as true or false instead of 1 or 0.
Empty relations arrays are not outputted for linked entities anymore, since linked entities never include relationships.
The iso_3166_1_codes, iso_3166_2_codes, and iso_3166_3_codes properties have been renamed to iso-3166-1-codes, iso-3166-2-codes, and iso-3166-3-codes, respectively. This only applies to lookup and browse requests; search requests already outputted these with hyphens.
The iso-3166- properties mentioned in the previous point are not outputted if they’re empty.

Some other changes to the web service have been made, but are considered additions (not changes to existing output), so hopefully shouldn’t cause any problems. You can review them in the changelog below.

Other Changes

An issue where entities deleted from the database (but still present in the cache) remained visible has hopefully been fixed. There are several other miscellaneous bug fixes linked below. Thanks again to Ulrich Klauer for his contributions. The git tag is v-2016-01-25.

Bug

[MBS-5676] – JSON relationships output doesn’t include target-type
[MBS-6166] – Deleted accounts can still have details edited
[MBS-7241] – Non-transactional cache means the cache can sometimes fail to delete entities that are gone at the database level
[MBS-7735] – ws/2: recording’s “video” flag inconsistent between xml and json
[MBS-7921] – Internal server error when requesting /ws/2/isrc as JSON
[MBS-8367] – ws2 JSON incorrectly returns non-included field as null value
[MBS-8396] – JSON output has no ordering key attribute for release group series
[MBS-8563] – Release & Release Group browse requests without type/status filters return results which contradicts the documentation
[MBS-8688] – Random tagged entity type display inconsistency in personal tag page
[MBS-8722] – Edit stuck trying to change the gender of a group
[MBS-8726] – Replicated updates don’t invalidate cache entries on slave servers
[MBS-8730] – Reordering of sub work parts causes unwanted reordering of main work parts
[MBS-8746] – JSON web service doesn’t distinguish between relationships not existing vs. not being loaded