Schema change releases for 2012

One of the issues raised at the last summit was that our customers could use more time to prepare for schema change releases, since they require engineering effort on their part. In an effort to meet our customer’s needs we’re going to a set schedule for schema change releases. Going forward we’re going to have two schema change releases per year: On or about 15 May and on or about 15 October.

We’ve picked these two dates as dates that have the least amount of impact from holidays and people taking holidays. Most companies have normal working schedules around these dates, which should allow companies to dedicate the required resources to handle our schema changes.

However, we have one significant change that we need to push out on a more timely basis than May of next year. For that reason we’re going to plan a one-time exception to our new schedule for 12 January 2012. You can see the tickets we’ve scheduled for release in January in this schema change milestone.

We’ve already created release versions in Jira for all of the schema change releases in 2012. As we go through the year we’re going to add tickets to that milestone. Of course, we’re going to make lots of noise as these schema change release dates approach. We’re going to post a list of tickets that will be included no later than a month before the release.

Any questions? Ask them in the comments!

Search server update 2011-12-08

We’ve just updated our search server with the latest changes and bug fixes!

This update to the Search Server finally solves the search for artist “!!!” problem properly, should now be able to find just about any artist, release, etc that contain any crazy character combination. Also you can now search for number of releases in release groups and artists with unknown gender. The code base has now been updated to Lucene 3.4, which was the latest available when these changes were made (its at 3.5 now).

Thanks for your hard work on this Paul!

Bug

  • [SEARCH-33] – Search needs to find a preposterously bad label name: !"@.*!%
  • [SEARCH-51] – Searching for certain characters returns no results, even if they’re valid.
  • [SEARCH-131] – When search for Unknown country it returns <country>UNKNOWN</country> it should return nothing
  • [SEARCH-134] – Annotation search can’t filter by type for release groups

Improvement

  • [SEARCH-93] – Update Search Index Code to Lucene 3.4
  • [SEARCH-119] – Allow artist search for gender:unknown
  • [SEARCH-124] – Allow searching release groups by # releases
  • [SEARCH-128] – Make Search Server use mmap by default

Server update 2011-12-05

Another update has just gone out, a few days later than planned. This is mostly a bug fix and minor improvement release. Sadly, when deploying the update we broke the release editor for anyone who was editing during the server switches – something we didn’t anticipate. Sorry! Here’s what’s changed:

Bug

  • [MBS-2371] – Can’t type in the basic tracklist editor
  • [MBS-2785] – Release Editor messes up release artist multi-artist credits
  • [MBS-3152] – Tracklist duplication with sub-second track duration differences
  • [MBS-3428] – If seed new release with track artist ids it doesnt properly resolve the artist ids
  • [MBS-3471] – Wrong display of RG in the Edit Note tab of the RE
  • [MBS-3498] – Editing tracklists on any release which has been through an artist split fails.
  • [MBS-3601] – Incorrect "Artist as credited" displayed and impossible to edit track artist in release editor
  • [MBS-3804] – Entering "Edit barcodes" edit should not be possible if nothing is being changed
  • [MBS-3808] – Internal server error searching for edits
  • [MBS-3813] – Release editor gets stuck on tracklist tab saying there are errors when there are none
  • [MBS-3814] – Guess case broken in the RE
  • [MBS-3816] – Track parser displays "This is placeholder text" instead of instructions (or nothing)
  • [MBS-3831] – "There were one or more errors, please check the following fields: " after changing the capitalization of one letter
  • [MBS-3836] – Tracklist tab showing only the first track when importing a CD stub
  • [MBS-3849] – RE: Can’t import FreeDB releases through Add Disc
  • [MBS-3858] – Tracklist duplication with equal sub-second track durations
  • [MBS-3864] – importing a cd stub only gives you the first track in the track editor
  • [MBS-3908] – Edit numbers in edit notes should not be parsed when they are on different lines

Improvement

  • [MBS-89] – Subscribed artist e-mails should be optional
  • [MBS-2924] – timeline.js should use MB.html
  • [MBS-3239] – Subscription emails > Option to receive weekly
  • [MBS-3356] – Seeded add release (copied from existing release) doesnt link tracks to recordings of copied release
  • [MBS-3684] – Improve sorting of the "Releases with superfluous data tracks" report
  • [MBS-3743] – Move the trackparser (basic view) on the tracklist tab to the "Add Disc" dialog.
  • [MBS-3892] – Display artist disambiguation comments on mouse over

New Feature

  • [MBS-3899] – Report: Discogs pages attached to multiple release groups

Task

  • [MBS-3911] – Remove all "DeferredUpdate" code

Sub-task

  • [MBS-3733] – Add create/get session logging to release editor, to debug MBS-3379 / MBS-3590.

Have Your Say In Scheduling

In an effort to bring the community into our development process more, I (ocharles) have built a new little application to help the MusicBrainz community decide exactly what we should focus on. I present to you, the Scheduling Game!

The Scheduling Game
The Scheduling Game

The idea behind this little application is to let the community quickly scan through open issues and decide when they believe they should be fixed. Each editor is presented with 3 options:

  1. Within 3 months. This ticket must absolutely be fixed within 3 months from now.
  2. Within 12 months. This ticket should be fixed within a year from now. This means the ticket should be fixed within the year, but is not critical for the next 3 months
  3. Unscheduled. This issue is not pressing and can be fixed much later.

Later, when issues have been voted on by enough people, the developers will try and decide how to schedule work to best fit the needs of the community.

A few disclaimers… Firstly, this is just an experiment! We don’t know how well it will work, though initial testing has shown it to already be quite useful. Secondly, it’s not really much of a game, but I wanted to give it a cute name (I know how picky everyone can be!). Currently only a subset of open tickets are available for voting on – specifically tickets in the “NGS + 1” and “Post NGS” milestones on JIRA, as these milestones are in need of a sorting.

The scheduling game is currently live on my server at http://scheduling.ocharles.org.uk/. You can have a look at the current (anonymous) votes at http://scheduling.ocharles.org.uk/results. If you have any questions or comments, you can reach me in IRC as ocharles or email oliver@musicbrainz.org. Happy voting!

Server update 2011-11-21

A few days behind on this one, but we’ve got this release out. This release has one controversial change, which I want to discuss in this blog post.

Displaying Relationships on Release Pages

A lot of people want this, and we’ve shipped something that adds this functionality. However, a lot of people think this is too verbose, and to a point I agree – there’s a lot of information there that can be displayed better. However, work on this issue has been on and off for almost the past 2 years. We’ve got a solution here that while far from ideal will get this ball rolling.

This is an unpopular move on my half, but I’m asking users to please try and bear with us until the next release (scheduled in for 5th December), which is just over a week away. Hopefully having this fairly messy display so open will raise enough discussion on how exactly to move forward. If you really must see it gone, you can use AdBlock or user styles in your browser to hide the dl.ars element.

Changes in this Release

  • [MBS-2067] – CDStub not displayed in the "Matching CDs" section on CDTOC lookup page
  • [MBS-3668] – all edits including "Auto-edits" are being tallied as "Accepted" edits in editor profile
  • [MBS-3688] – Wrong coverart on all-zeros barcodes
  • [MBS-3718] – Inline search has problems with search terms with " (double quotes) and colons in them (and maybe others)
  • [MBS-3734] – Google indexes MusicBrainz search result (and some other) pages it shouldn’t
  • [MBS-3754] – Splitting an artist shouldn’t be an auto-edit
  • [MBS-3755] – Importing from FreeDB puts the artist name in the wrong field
  • [MBS-3759] – Auto-editor election isn’t showing own vote (instead: private)
  • [MBS-3764] – Typo on artist merge page
  • [MBS-3768] – Error when editing URL into one that already exists
  • [MBS-3777] – Musicbrainz breaks https://secure.wikimedia.org link in edit notes
  • [MBS-3786] – Entities with tag count on top of tag view page is incorrect

Improvement

  • [MBS-2979] – Enable CORS (Cross-Origin Resource Sharing)
  • [MBS-3105] – Country/Language/Script statistics should include a statistic for those with none set.
  • [MBS-3752] – Making edit note for "Add release" mandatory
  • [MBS-3767] – Distinguish between Discogs master and release URLs on release pages
  • [MBS-3778] – Separate release group links from release links on release pages

New Feature

  • [MBS-496] – Display recording relationships on a release page
  • [MBS-3761] – Add songfacts to the sideboard

Server update 2011-11-14

It’s been a while, but we’ve finally got a new server release out! This one didn’t go perfectly smoothly, and we had (more, i’m afraid) troubles with the release editor. We’ve got a fix for this out, so editing should be at least as stable as it was before. As always, please do continue to report any bugs you find. Here’s what’s changed:

Bug

  • [MBS-761] – Release has ASIN but no cover art ("No cover image available.")
  • [MBS-1169] – Musicbrainz website is missing HTML escaping on user entered data
  • [MBS-2529] – Orphaned release group and recordings after rejecting or cancelling a release/medium
  • [MBS-2590] – Unused URLs not being removed
  • [MBS-2847] – Merging artists moves the comment from the old artist to the new one
  • [MBS-3102] – Changing release artists automatically changes all track artists
  • [MBS-3305] – Artist missing in edit medium edit
  • [MBS-3451] – Unterminated link on CD Stub page causes "Overview" and "Edit" tabs to display incorrectly
  • [MBS-3466] – otherlookup for non-existent MBID throws ISE
  • [MBS-3522] – Bad gateway errors in Add Missing Entities trying to add large VA releases without locking down artists
  • [MBS-3656] – Clicking text (Yes/No/Abstain/None) on the most recent open edit changes votes on all edits in that page.
  • [MBS-3657] – Medium order not allowed when merging releases
  • [MBS-3675] – Modbot failing edit relationship edits with "This relationship has already been deleted"
  • [MBS-3677] – Recording page doesn’t highlight releases with pending edits or show disambiguation comments
  • [MBS-3689] – Tagger button on tag lookup for recordings confusingly leads to "non-album tracks"
  • [MBS-3715] – Fix the wording for YouTube videos on the sidebar
  • [MBS-3719] – Format/display of recording search results is broken for standalone recordings
  • [MBS-3741] – Incorrect artist_credit.name

Improvement

  • [MBS-1766] – Display lyrics and score links in work sidebar (in the External link section)
  • [MBS-2478] – Pages for releases should also show ARs from the release group they are contained within.
  • [MBS-2942] – Warn users that FreeDB indexes are only updated monthly
  • [MBS-3192] – Display release group URLs also on release pages
  • [MBS-3495] – Make IMSLP links look like the Wikipedia ones
  • [MBS-3508] – Indicate if a release group contains only one release in release group
  • [MBS-3713] – Add JS autoselect for the SoundCloud relationships
  • [MBS-3714] – Add cleanup for YouTube video URLs
  • [MBS-3747] – Create script to split arranger relationships

New Feature

  • [MBS-71] – Add auto-editor elections

Task

  • [MBS-3697] – MusicBrainz needs to comply with new changes of Amazon Product Advertising API taking effect on October 25, 2011
  • [MBS-3710] – Tranclude "People" instead of "MusicBrainz_Bio"

Dear python-musicbrainz/0.7.3 application, we need to talk!

An application that uses our python-musicbrainz/0.7.3 client library has been putting undue load on our servers all at once. This application looks up something at MusicBrainz at 03:00UTC causing our servers to be overloaded at that time each day.

To protect our servers from being overloaded we’re going to block this application from 3:00 UTC – 4:00 UTC. We’re hoping that this will alllow us to identify the application and start a dialog with the application authors. Once we have established communication with the authors and worked up a plan to fix this, we’re going to release the block.

We really dislike blocking applications, but if applications are being inconsiderate of our resources, we’re left with few options. We hope to hear from the application authors soon so we can resolve this issue. Also, we’re moving forward with our plans to require User-Agent strings that properly identify applications using our service to fix this problem going forward.

If you are the author of said application, please leave a comment with information on how we can get in touch with you.

Release editor service interruption

In an effort to mitigate/fix MBS-3379 we need to restart the service that keeps the session information for our release editors. We’re going to do that tomorrow Saturday October 28 at Noon PDT, 3PM EDT, 8PM London, 9PM Amsterdam. If at this time you have a release editor open, submitting your edits will fail and you will need to start your edits over again.

Sorry for the inconvenience.

Looking for sponsors to purchase two development machines

The MusicBrainz community has two very active contributors (navap and reosarevok) who do not have and cannot afford to buy decent development machines/laptops. MusicBrainz is carefully watching its finances currently and can’t really afford to spend money on this right now, so I’m curious to see if the community would come together to support these two awesome contributors.

Navap has done a lot of editing and web site improvements (a series of 8 or so nitpick patches that greatly improved the look/feel of NGS) and countless other tasks to improve MusicBrainz. Reosarevok is a very active editor and has recently become our wiki god; he is also wishing to help out more with the development. Both of these two awesome contributors could be more productive with decent machines.

We’re looking to buy two $500 Linux capable machines that have a couple of cores and about 4GB of RAM each. Would you please consider chipping in to help our contributors? You can make a cash donation (please let us know your donation should go towards laptops) or even send a laptop to our contributors. If you’d like to help out with a donation of an actual machine, please leave a comment in the blog and I’ll get in touch with you.

Thanks!

Web service user-agent string blocking reminder

I would like to remind Web Service users that on 16 November we’re going to block generic User-Agent strings from accessing our web service. Earlier we said:

The User-Agent string needs to identify the application and the version of the application that is making the request; having a generic User-Agent string like “Java/1.6.0_24″ or “PHP/5.3.4″ does not allow us to properly identify the application making the requests.

IMPORTANT: 6 Months after we release NGS (Nov 16th) we’re going to start blocking common generic User-Agents strings, so please make sure that you send us a proper User-Agent header as part of your request.

You have been warned. 🙂