Search Server Update, 2011-09-28

On Wednesday we rolled out an upgrade of our indexed search servers. These search servers are used for search in both the web service and website. This release includes some fixes and improvements added to make it easier to handle some particular search cases, plus few other fixes. A fairly critical bug with respect to work searching was came up yesterday after this update, but we’ve released a hot-fix patch to solve that problem. Here are the release notes:

Bug

  • [SEARCH-82] – Relation Type in Work relation list is reported as {xxxx}{yyyy}zzz
  • [SEARCH-104] – Querying "code:XXXXX" in a label search returns no results.
  • [SEARCH-106] – target-type attribute missing in relation-list elements
  • [SEARCH-110] – Web service is returning ‘disambiguation’ element in release groups
  • [SEARCH-112] – Label codes are indexed as strings
  • [SEARCH-113] – Label aliases don’t produce the expected results
  • [SEARCH-116] – Freedb import chokes on entries with multiple ids
  • [SEARCH-121] – Recording search only searches on recording artists not track artists if different

Improvement

  • [SEARCH-114] – Index unset artist countries as unknown
  • [SEARCH-115] – Allow advanced search for releases with zero mediums

New Feature

  • [SEARCH-90] – Search for standalone recordings

Our new wiki warden: Nicolás Tamargo (reosarevok)

After too many moons of neglect, we finally have a new wiki warden who is going to put some serious effort into sprucing up our wiki. Nicolás Tamargo (user reosarevok) has been promoted to be our new wiki god. His first task is to clean up dead pages and to get rid of a lot of cruft; then he’ll work on improving some doc pages that have been long out of date. Hopefully soon we can get to a state where the wiki is useful, rather than filling us with a sense of dread. 🙂

If you’re interested in helping reosarevok with the wiki, please let him know by leaving a comment here or contacting him directly.

Thanks for taking on our very messy wiki, Reosarevok!

Record traffic, new server in NGS rotation

We’re currently seeing record levels of traffic right now. Since the beginning of August our traffic has gone from 9M hits per day to 14M hits per day today, which is a significant increase in traffic. Fortunately our servers were able to handle the extra load, but we started getting near capacity.

Yesterday I started working on taking one of the servers from the classic site and moving it into the NGS cluster. We finished this a couple of hours ago and we now have a maximum limit of 227 queries per second (qps).

If you find any spurious troubles with the site in the next day or so, please open a ticket.

NGS Bug fixes 2011-09-21 and the New "Split Artist" Feature

Apologies for being a few days late here, but we’ve just finished updating the servers with the latest set of bug fixes and new features. This release brings a new feature that people have been asking for since NGS – the ability to “split” an artist into artist credits.

Split Artist

With NGS, we had the ability to link multiple artists to recordings/releases/etc. While our migration script handled most cases, there are some cases when it was unclear how to automatically split artists, and we opted to leave that artist unchanged. While it was possible to manually edit references to this artist, it was painful. We now have a convenient way to make this process easier. In the sidebar of artists there is now a “split into separate artists” link:

The new "split into separate artists" feature
The new "split into separate artists" feature

Clicking this link will present you with a form where you can choose the new set of artists. For example, here’s me splitting the single “Alix Perez & SpetraSoul” artist into the correct separate artists:

The "split artists" form
The "split artists" form

As with the other types of editing in MusicBrainz, this edit has to be voted on by other editors before it is applied.

The split artists edit type can only be used on artists that have no relationships, other than “collaboration” relationships. If the artist does have collaboration relationships, edits to remove all collaboration relationships will be entered at the same time. Note also that this will not remove the now empty artist entirely – that will happen when ModBot does its daily cleanup.

Bugs

We’ve continued working through the bugs as well, and here are the tickets that have been closed in this release.

  • [MBS-1133] – Don’t try to delete relationship types that are still in use
  • [MBS-1343] – Add AR instrument search function has problems with accented letters
  • [MBS-1380] – An URL ending in closing parenthesis gets broken when listed in edit notes
  • [MBS-1396] – Allow for URLs longer than 255 chars
  • [MBS-1411] – Don’t lowercase relationship attributes
  • [MBS-1520] – Tag lookup page has odd search results
  • [MBS-1879] – Remove the non-functional /search/plugins page
  • [MBS-1934] – Inconsistency: “see other versions of this release” vs. “see release group”
  • [MBS-1936] – Historic MusicBrainz::Server::Edit::Historic::EditArtist incorrect: doesn’t include artist names, always use “[removed]”
  • [MBS-1950] – Artist Sortname is shown instead of Artist Name in historic edits
  • [MBS-2047] – Email sender addresses have regressed
  • [MBS-2195] – Don’t allow submitting relationships where the relationship type has no description
  • [MBS-2335] – Other Lookup ISWC search should strip spaces
  • [MBS-2495] – Auto-edit status in edit documentation is misleading
  • [MBS-2529] – Orphaned release group and recordings after rejecting or cancelling a release/medium
  • [MBS-2911] – URL decoding should not decode non-utf-8 strings
  • [MBS-2934] – “Change release group” edit doesn’t appear in artist’s edits
  • [MBS-2944] – Vote on all edits control should only be shown to auto-editors
  • [MBS-3226] – Multi-artist credits with no artist selected produce very confusing behavior
  • [MBS-3227] – Lookup TOC page incomplete for logged out users.
  • [MBS-3269] – Track #s get automatically added to track titles every time the tracklist tab is opened
  • [MBS-3274] – Editing track names using with the basic editor removes track artists and disassociates recordings
  • [MBS-3278] – Warning about using various artists as track artist flashes on screen
  • [MBS-3347] – When trying to add cover art from disallowed sites, no error is shown
  • [MBS-3351] – ws/2 stacktrace browsing release by label
  • [MBS-3358] – “Open Edits” and “All Edits” tabs not marked as open
  • [MBS-3373] – URL cleanup stuff doesn’t always run in time
  • [MBS-3381] – Orphaned release groups/recordings created by release editor should be automatically removed
  • [MBS-3391] – Don’t allow to enter “performs instrument” without instrument
  • [MBS-3399] – No edit links for relationships in the appearances section on the artist relationships tab
  • [MBS-3402] – Do not allow subscriptions to “Various Artists” or “Deleted Artist”
  • [MBS-3403] – “Change release group” edit doesn’t show up in edit history / open edits of target release group
  • [MBS-3405] – Remove leading / trailing / double spaces in titles
  • [MBS-3434] – High DQ Release allows Auto-edits + incorrect edit conditions for merge
  • [MBS-3436] – “Wiki Version” column on transclusion page shows ERROR
  • [MBS-3443] – Inconsistency in sorting of “Edits by subscribed editors” ≠ “Edits for subscribed entities”
  • [MBS-3445] – Cannot search edits by editor name
  • [MBS-3453] – “Related works” on recordings page shows same work and AR set twice if there are two work<->recording ARs for the same work/recording pair
  • [MBS-3464] – Do not redirect http://www.musicbrainz.org to musicbrainz.org for /ws requests

Task

  • [MBS-1914] – Run script to remove “later/earlier release of” ARs

Improvement

  • [MBS-2669] – Filter edits by language
  • [MBS-2670] – Filter edits by artist country

The Git tag for this release is v-2011-09-21-ngs-bug-fixes. Happy editing!

MusicBrainz Summit 11: Please come join us in Rotterdam!

I wanted to remind people that we’re finally having a summit in October. We’ve rented an apartment in Rotterdam, Netherlands as our venue from 13 October to 18 October. On the afternoon of October 13 and most of October 14 we’re going to have a two day-ish hackathon for hacking on various MusicBrainz related things. On the evening of the 14th we’re going to have a social meet and greet event at the apartment. The summit itself will be held on the 15th and 16th, following an unconference like format where the attendees get to set the agenda. Most people will depart Rotterdam on the evening of the 16th. On Monday the 17th, our final day in the apartment, we’re going to have a day focused on MusicBrainz’ commercial users to discuss issues from a commercial perspective.

If you’re a MusicBrainz hacker/supporter and would like to attend, we’re going to cover lodgings and food for a limited number of people through the generous sponsorship of last.fm and MusixMatch. Our budget is not vast, so if you’re interested in attending the event, please sign up on our the summit wiki page. You’ll be responsible for your own transportation to Rotterdam.

Warp, Ollie and myself will be at the apartment the entire time and we’ll be bringing some creature comforts like game consoles and other things to keep ourselves entertained when we’re not discussing or hacking on MusicBrainz.

Please come join us in the Netherlands!

NGS Bug fixes, 2011-09-12

Things in MusicBrainz land are finally starting to return to normal, with a succession of holidays all final over and everyone getting back to work. We sadly missed the last release of August entirely, but have combined it with the latest release, which was deployed onto servers yesterday. Here’s what’s changed:

Bug

  • [MBS-987] – Database import doesn’t work without editor data
  • [MBS-1543] – Add credits to the images in the release editor
  • [MBS-1670] – UX, Can’t add new artist if an artist is already selected in artist credit
  • [MBS-1977] – ModBot is unable to close some edits
  • [MBS-2030] – Historic "Add release" edits: artist displayed as "[removed]"
  • [MBS-2184] – Indexed inline search find no recording for 夜曲 but there are several
  • [MBS-2498] – Description for failed vote is inaccurate.
  • [MBS-2641] – Release editor doesn’t handle copied artist credits very well
  • [MBS-2710] – merging recordings shouldn’t kill release edits
  • [MBS-2817] – Add Missing Entities not displaying correct search results for many artists
  • [MBS-2823] – Edit closed as failed but was actually accepted
  • [MBS-2829] – Fuzzy CD lookup doesn’t work if you don’t exclude CDStubs
  • [MBS-2902] – When editing release, ".[number]" at end of artist name is parsed as track time.
  • [MBS-2945] – Musicbrainz should report a sensible error when add release fails because of invalid data rather than just ‘Internal Error’
  • [MBS-3066] – Splitting to multiple artists using Credits fields only recognises first artist
  • [MBS-3095] – http://www.musicbrainz.org/ should redirect to http://musicbrainz.org/
  • [MBS-3113] – Invalid URLs shouldn’t be accepted
  • [MBS-3147] – Cover art script not working properly
  • [MBS-3209] – Recordings need an equivalent to the tracklist’s ‘add missing entities’
  • [MBS-3224] – Internal Server Error in the release editor after going back in the browser
  • [MBS-3243] – Unable to have a track named "0" in the track list
  • [MBS-3258] – column width in opera too small
  • [MBS-3272] – InitDB can fail if the locale of the machine isn’t set correctly
  • [MBS-3275] – Internal server error when adding a new release
  • [MBS-3280] – ISE on tracklist tab of release editor (?)
  • [MBS-3287] – Looking up an existing release group when adding a release should disable the type field
  • [MBS-3289] – On the Release Information -> Tracklist and Release Information pages the Join phrase is not shown in the Preview
  • [MBS-3297] – A space in track# field in advanced tracklist causes ISE
  • [MBS-3298] – Rendering the timeline eats all my CPU and takes forever
  • [MBS-3301] – Adding tracks to a release fails with "This edit changes recording IDs, but some of the recordings no longer exist."
  • [MBS-3303] – Columns are the wrong width
  • [MBS-3306] – Adjust flags and delete tabs are shown, but don’t work
  • [MBS-3312] – Edits fail because recordings do supposedly not exist anymore but in fact still exist
  • [MBS-3344] – Stop closing edits as "Error"
  • [MBS-3355] – When match track with unknown duration to an existing recoridng it doesnt set the tracktime
  • [MBS-3365] – Entries listed twice on artists work pages
  • [MBS-3366] – "There are not currently any relationship types available between works and recordings." error when trying to link recordings to works
  • [MBS-3369] – Internal Server error on Edit Types documentation
  • [MBS-3378] – Edit label not showing any information
  • [MBS-3380] – Attempt to add duplicate artist via Release Editor causes Internal Server Error
  • [MBS-3400] – Expired merge releases edit not being applied.
  • [MBS-3404] – Move disc ID edit type: incorrect source release

Improvement

  • [MBS-2188] – "Comment" is confusing people
  • [MBS-2256] – UX, the current artist credit interface does not discourage changing the credit for various artists
  • [MBS-2839] – Artist work list > Better design: don’t split by "categories" (composer, lyricist, …) and indicates artist role for each artist related to work
  • [MBS-2899] – Release editor’s "Release Duplicates" tab does not suggest any releases if the release title has been translated away from the actual title
  • [MBS-3211] – allmusic and secondhandsongs not auto-detected

New Feature

  • [MBS-3261] – Add support for "deleting" user accounts

The tag for this version is v-2011-09-12-ngs-bug-fixes.