Category: Development

Server update 2012-03-19

We’ve just finished pushing out a new server update, that addresses a few minor bugs. We initially planned to roll these changes in with the release of our Cover Art Archive, but sadly this project needs a bit longer. Many thanks to Johannes Weißl, Wieland Hoffman, Ian McEwan and the MusicBrainz team for their work on this release.

Bug

  • [MBS-3897] – Missing space in template for ARs w/disambig
  • [MBS-4291] – Default install looks for compiled js/css.
  • [MBS-4368] – Release editor: track parser is broken
  • [MBS-4383] – Statistics page is showing "Active Last Week" stats twice, leaving out "Edited Last Week"
  • [MBS-4391] – Track parser broken
  • [MBS-4399] – Medium titles not displayed when attaching a discid to a medium
  • [MBS-4403] – DBD::Pg::st execute failed: ERROR: new row for relation "recording" violates check constraint "recording_length_check" at lib/Sql.pm line 396
  • [MBS-4404] – /ws/1 uses the wrong language code
  • [MBS-4407] – Edit artist alias should fail if edited alias is deleted

Improvement

  • [MBS-1363] – Add threading headers to mod note emails
  • [MBS-1473] – Report: unlikely language-script combinations
  • [MBS-3313] – New/old edits wording in edit search is misleading
  • [MBS-4044] – Cleanup Discogs URLs
  • [MBS-4284] – Normalize URL encoding for specific sites

A Git tag will be pushed out later. The commit sha is 03343aa460d1.

Preparing for the May 15th schema change release

For our last schema change release we had a ton of issues around which tickets we should address and which ones were properly defined for us to work on. I’d like to make this a lot more clear for the next go round; here is what we’re going to do:

Starting today and for the next two weeks, we’re going to seek people to be the champion (sponsor) of a ticket. If you feel strongly about a schema change ticket getting taken care of, you should consider championing this ticket. Once you’ve decided to do adopt a ticket, you should assign the ticket to yourself.

Then, over the next two weeks it will be up to you to do the following:

  1. Derive consensus around the core concept of the ticket. If you go through the process of working up a ticket, but no one agrees with what you’re proposing, you’ve wasted your time. Make sure that you get buy in from others in the community. For instance, if Nikki doesn’t like it, chances are its not going to fly. 🙂
  2. Ensure that the ticket clearly states what needs to be done to implement the ticket. The ticket should essentially become or link to a requirements document. This requirements document should explain what the new feature should do. It should not explain how it should be done — we should leave the how to our developers who are going to implement the feature.
  3. Provide as much supporting documentation as you can. Mock-ups for UIs are deeply appreciated (even if they delve into the how realm of things) and very useful for meaningfully discussing these tickets.
  4. Have the ticket reviewed by a developer for clarity and completeness, then address any issues said developer may raise.

On 19 March, we’re going to look at the list of tickets that people have taken on and choose the ones that are clear enough to move forward. If you’ve done all the work outlined above, the chances are good that your ticket will be chosen to move forward. If your ticket is chosen to move forward, there will be more questions that the developers will raise — hopefully those can be tackled in the space of a week. After that we will take all of the well defined tickets and schedule them for implementation. All the other tickets that are not clear to implement will be rejected and will have to make another pass though this process in the autumn.

If you’re still interested, here is the list of schema change tickets that should be considered for this.

Server update 2012-02-27

Sorry for being a whole week behind the scheduled update time! However, we’ve got a huge release here, which will hopefully make up for the delay!

Many thanks to Calvin Walton, Johannes Weißl, Ian McEwen, Michael Wiencek and the MusicBrainz developers for all their work on this release.

Bug

  • [MBS-1818] – UX issues with "Add Disc" feature
  • [MBS-2363] – Release editor: Release Duplicates tab doesn’t keep selection…
  • [MBS-2889] – Advanced tracklist edit: artist field is disabled by default for multi-disc VA releases
  • [MBS-2967] – Newly created artist not displayed in edit notes diff when changing release artist
  • [MBS-3027] – License attribute isn’t hidden when the relationship type is automatically changed
  • [MBS-3290] – User:Whatever links in /doc/ should point to wiki
  • [MBS-3659] – Can’t call method "statistic_names" on an undefined value
  • [MBS-3776] – YouTube Streaming autoselect doesn’t load the attribute
  • [MBS-4010] – Duplicate artist credits
  • [MBS-4043] – URL relationship type description not updating
  • [MBS-4133] – Internal server error editing recording
  • [MBS-4141] – "Add missing entities" tab breaks if you have to add a new label and have an existing artist
  • [MBS-4213] – Internal Server Error when adding a new release (Validation failed for ‘Maybe[Int]’ with value 22439.2)
  • [MBS-4269] – Wrong expansion for youtu.be URLs
  • [MBS-4280] – Translation.pm doesn’t check that $msgid is defined before regexing it
  • [MBS-4285] – Empty title returned instead of "0"
  • [MBS-4287] – Artist Country condition doesn’t load on edit search
  • [MBS-4288] – Recording tab in the RE no longer shows track ACs
  • [MBS-4290] – Regression on javascript inline search’s keyboard events
  • [MBS-4292] – License problem with excanvas.js
  • [MBS-4293] – "Get wiki versions" on transclusion table does nothing
  • [MBS-4295] – FreeDB importing chokes on double quotes
  • [MBS-4297] – Amazon referrer stuff not included in sidebar links
  • [MBS-4311] – Sort unlinked pseudo-releases report
  • [MBS-4333] – Release editor > Importing from CDStubs is broken
  • [MBS-4351] – URL Relations with ampersands in the URL aren’t being escaped in XML returned by the WebService API
  • [MBS-4352] – JS URL cleanup for recording licenses

Improvement

  • [MBS-942] – Link back to editor’s collections on the collection page
  • [MBS-1389] – Accept "X.XX" track times
  • [MBS-1435] – Show ISRC / ISWC in normal search
  • [MBS-1657] – "Change vote + add note" in edit page
  • [MBS-1961] – Display more information on "merge work" edits
  • [MBS-2086] – Aliases: separate localised aliases from other aliases
  • [MBS-2120] – Track parser needs to understand – (en dash) as an artist/title separator
  • [MBS-2392] – Track artist {credits} not shown on CD lookup
  • [MBS-2573] – Make add alias default to the current name
  • [MBS-2808] – Annotation wiki formatting syntax help is missing
  • [MBS-2957] – Hide duplicate labels and cat#s in recording / RG views
  • [MBS-3371] – Report: releases with non-consecutive medium numbers
  • [MBS-3398] – Detect "M" in front of track numbers
  • [MBS-3737] – Change the colour of the artist names on the advanced tracklist
  • [MBS-4044] – Cleanup Discogs URLs
  • [MBS-4135] – Display collection owner in release page’s collection tab
  • [MBS-4262] – gzipping of dynamic content could use extension to javascript and JSON
  • [MBS-4332] – "Add medium" edit display: Release and position fields should be at top rather than at bottom

New Feature

  • [MBS-3938] – Add a report to show artists that may have disambiguation strings in the artist name
  • [MBS-4081] – Report: pseudo-releases without transl*tion relationships
  • [MBS-4362] – Report: Recordings with Creative Commons Download relationships

Task

  • [MBS-2836] – Trim annotations in database
  • [MBS-3416] – Automatically delete release annotations that consist only of white space

A Git tag should be live later, but the commit is b5f45992a24bf.

Server Update, 2012-02-06

We just finished pushing out an update last night, and sadly it didn’t go particularly smoothly. As part of ongoing efforts to try and fix the release editor, we temporarily made everything worse and ended up losing quite a lot of session data. I’m profusely sorry that this didn’t go better! We think things are stable again now though, so if you do see any bugs that you haven’t encountered before, please do make sure to let us know over at the bug tracker. Other than that, there were quite a few internal changes to our production setup in this release – we moved to a more modern Plack based setup and used our recent Carton packaging work. Here are the other bugs that got fixed. Thanks to Ian McEwan and Michael Wiencek for helping make this release happen!

Bug

  • [MBS-2112] – Using space as join word between two artists in a credit ends up with no space between them
  • [MBS-3677] – Recording page doesn’t highlight releases with pending edits or show disambiguation comments
  • [MBS-3949] – Lookup box not working on work-URL form
  • [MBS-4248] – Timeline graph doesn’t work due to poorly-escaped strings in JS
  • [MBS-3680] – Error 502 Bad Gateway loading certain releases from the webservice

Improvement

  • [MBS-343] – Lookup by MBID in inline searches
  • [MBS-4273] – WS should use ETags in order to save clients bandwidth (and us too)

New Feature

  • [MBS-4210] – Use separate robots.txt for staging and production servers

Task

  • [MBS-4189] – Display language zxx ("No linguistic content") in release editor form

Server update, 2012-01-26

We’ve just pushed out the last MusicBrainz web server update out for this month. Here are the bugs we’ve squashed, and the new features and improvements that have been added. As always, keep helping out with scheduling at the scheduling game, and we’ll do our best to get the important things fixed as soon as we can! Thanks to Ian McEwan, Lukáš Lalinský, Calvin Walton, Johannes Weißl and the MusicBrainz developers for their work on this update.

Bug

  • [MBS-2690] – Release editor reports incorrect "Reorder mediums" edits
  • [MBS-3692] – Cover image from amazon.es is not displayed
  • [MBS-3798] – Cover art not working for amazon.cn
  • [MBS-4040] – "Add URL relationship" doesn’t autodetect YouTube channel URLs
  • [MBS-4150] – Edit recording edits include artist credits even when they didn’t change
  • [MBS-4174] – Changes to track times don’t allow to propagate to recordings
  • [MBS-4177] – Don’t add "musicbrainz" to the search path for tests
  • [MBS-4181] – Date field doesn’t display nice error message
  • [MBS-4184] – label releases list sorted wrong
  • [MBS-4185] – Release Editor should not offer to update recordings when creating new ones
  • [MBS-4198] – "Link type" in edit search should be "relationship type"
  • [MBS-4199] – Search criteria when searching edits by relationship type breaks after initial search
  • [MBS-4202] – The "Merge releases and recordings" recording diff always thinks artists are different
  • [MBS-4211] – Copy all track changes crashes with new recordings
  • [MBS-4222] – When merging releases, I’m asked to pick a "work"
  • [MBS-4231] – JS URL cleanup recognizes more URLs than it should

Improvement

  • [MBS-2037] – Unify all references to memcached servers in lib/DBDefs.
  • [MBS-2799] – RE: Add the option to indicate a release has no barcode
  • [MBS-4178] – Languages/Scripts statistics page should exclude unused/unavailable items
  • [MBS-4182] – Add RDFa attribute for artist IPI codes
  • [MBS-4187] – Display language and script statistics next to each other
  • [MBS-4191] – Stop auto-transcluding the Proposal namespace in wikidocs
  • [MBS-4195] – Improve Google descriptions
  • [MBS-4243] – Use EWMA to smooth rate of change statistics graphs
  • [MBS-4247] – Clarify language/script statistics to explain why some languages/scripts aren’t visible

New Feature

  • [MBS-3273] – Avoid memcached clashing when running in shared environments
  • [MBS-4244] – Add medium format distribution statistics
  • [MBS-4245] – Add release format distribution statistics
  • [MBS-4246] – Add artist credit distribution statistics

The tag for this release is v-2012-01-26-ngs-bug-fixes.

libmusicbrainz 4.0.0 released!

Andy Hawkins says:

In conjunction with Lukas I’ve been working on a complete rewrite of libmusicbrainz to support the new Web Service v2 released alongside NGS. I’m happy to announce the official release of this new library (libmusicbrainz4).

The library is written as a relatively simple parser for the results from the Web Service, with as little logic as possible. This should mean it is easy to maintain to keep in step with any changes in the schema for the XML Web Service.

The main interface is a C++ class ‘CQuery’, that returns a Metadata object containing a parsed version of the response. There is also a C interface that loosely wraps the C++ classes.

Using the library requires a good understanding of the queries and responses to and from the Web Service. Download the library, or browse the online documentation or generate the documentation from the source with make docs.

If you find any issues, please report them using JIRA (under the project ‘libmusicbrainz’).

I will follow the musicbrainz-devel mailing list and the #musicbrainz-devel IRC channel on freenode in order to answer questions if at all possible.

Thanks.

Andy

2012-01-12 Schema change fix for replicated slaves

Sadly, our testing for replication failed to catch a problem with the instructions we posted yesterday. If, when running the replication scripts you encountered the below error, please read on!

Attribute (conn) does not pass the type constraint because: Validation
failed for 'DBIx::Connector' with value DBI::db=HASH(0x9e957d0)
(not isa DBIx::Connector)

We made some changes to our database connection routines recently, and the replication scripts were not correctly migrated. We’ve got these changes in now though, so to fix this problem, repeat the instructions in our previous post, but use the v-2012-01-12-schema-change-2 tag. This should correspond to commit eb89c2b51f79..., which you can verify by running git rev-parse HEAD.

Sorry about these problems!

2012-01-12 Schema change upgrade instructions for replicated setups

We just finished the latest schema change release! If you are running a full musicbrainz-server mirror, then follow these instructions for upgrading your installation:

  1. Take down the web server running MusicBrainz, if you’re running a web server.
  2. Turn off cron jobs if you are automatically updating the database via cron jobs.
  3. Make sure your REPLICATION_TYPE setting is RT_SLAVE
  4. Switch to the new code with git fetch origin followed by git checkout v-2012-01-12-schema-change-2
  5. Install any necessary perl modules by running perl ./Makefile.PL and sudo make installdeps
  6. Run ./upgrade.sh from the top of the source directory.
  7. Set DB_SCHEMA_SEQUENCE to 14 in lib/DBDefs.pm
  8. Turn cron jobs back on, if needed.
  9. Restart the MusicBrainz web server, if needed.

If you are running a mbslave mirror, check out the latest code and read the upgrade instructions in the README file.

Schema change, 2012-01-12

We’ve just finished pushing out the scheduled schema change update today. A slightly rocky experience, but we’ve managed to get everything ironed out now and our servers are back to their normal selves. Before, I get into release notes, a few notes for people who’d like to upgrade their non-replicated servers. For replicated servers, please see this post.

  1. First, update your repository to the v-2012-01-12-schema-change tag.
  2. Next, disable any MusicBrainz cron jobs you might have scheduled to run
  3. Run ./upgrade.sh provided in the root directory of your musicbrainz-server checkout.
  4. Update your DBDefs DB_SCHEMA_SEQUENCE configuration option to 14.
  5. Turn any cron jobs back on

And onto the release notes!

Bug

  • [MBS-2121] – Deleting track/row in advanced tracklist editor while artist credits "dialog" is open leaves it stuck
  • [MBS-2390] – It’s not really possible to assign a track to a recording because cannot see length of track in either tracklist or proposal list
  • [MBS-2620] – Most ‘edit’ edits do not check for conflicts
  • [MBS-3393] – App keeps DB connections open and doesn’t reconnect upon DB restart.
  • [MBS-3696] – Using Jamendo as License Relationship fails due to Javascript change to Cover Art relationship
  • [MBS-3972] – ISWC column on works page should be nowrap
  • [MBS-4046] – Releases with pending edits are not marked as such in label page
  • [MBS-4099] – The server should give a good error message when it can’t connect to the DB
  • [MBS-4103] – Problems building database extensions with Ubuntu 11.10
  • [MBS-4129] – "Edit artist" doesn’t change artist credit for credits that were unchecked by default but checked manually
  • [MBS-4132] – Rate limited pages send HTML as text/plain
  • [MBS-4134] – Javascript sorting broken on Country statistics page since comma has been introduced
  • [MBS-4136] – beta.mb: recording edits generated by release editor when updating recording information from track changes are not auto-edits
  • [MBS-4138] – Language-Script page in statistics broke
  • [MBS-4146] – Track times deleted if using tab key
  • [MBS-4149] – The added referrer stuff breaks Amazon UK links
  • [MBS-4157] – beta.mb: Edit recording edits seem to create a ‘edit release’ edit too
  • [MBS-4169] – CDToc page shows blank/empty catalog numbers separated by commas

Improvement

  • [MBS-1482] – Report: no language, no script
  • [MBS-2439] – Releases with multiple catalog numbers are shown multiple times on the Label page.
  • [MBS-2513] – Allow updating recording information when track information changes via the release editor
  • [MBS-3288] – Release group type isn’t shown when looking up a release group
  • [MBS-4105] – Warn when merging releases with track artists that don’t match

New Feature

  • [MBS-4147] – Display AcoustIDs using JavaScript

Task

  • [MBS-3672] – Remove the deprecated Live Sound Engineer relationship

Sub-task

  • [MBS-2676] – Filter by relationship type

Server release tomorrow, January 12: Noon PST, 3PM EST, 20:00UK, 21:00CEST

We’re going to push out a server release tomorrow, January 12, at Noon PST, 3PM EST, 20:00UK, 21:00CEST. We expect to be offline for only a few minutes, but we’re going say that we’ll be offline for about 15 minutes starting right around that time.

Once we complete the server release, we’re going to post the usual list of bugs fixed, changes applied and new features created. Stay tuned!