Server Update, 2013-06-25

We’ve just finished pushing out another version of MusicBrainz. Sorry that we’re a day behind on this one, yesterday we had a few tests that needed fixing before we could release. This release is mostly bug fixes and small improvements, and should solve problems with the “remember me” login feature working incorrectly. This upgrade might have lost a few remember me tokens, but if you are still seeing problems after a few days of the site, please leave a comment on MBS-6471.

Many thanks to Lukáš Lalinský, Michael Wiencek, Nicolás Tamargo and the rest of the MusicBrainz team for their work on this release. Here’s what changed:

Bug

  • [MBS-3427] – "Internal Server Error" submitting CD Stubs with invalid data
  • [MBS-5524] – Edit relationship type edits are missing information for identifying the relationship type
  • [MBS-5903] – JSON webservice doesn’t return release-list in collection query
  • [MBS-5958] – Release editor did something very very wrong
  • [MBS-6027] – JSON webservice returns asin: null for releases with ASINs
  • [MBS-6073] – Inaccurate range of credits when grouped at the bottom of a release page
  • [MBS-6101] – Redis should only be required if sessions are used
  • [MBS-6206] – Soundtrack Collector URLs should not be possible at the Release level
  • [MBS-6244] – Private data dump contains bcrypt column and can’t be imported
  • [MBS-6261] – "Date unknown" is inconsistent with our handling of unknown values elsewhere
  • [MBS-6296] – Area alias links are shown for non-location-editors
  • [MBS-6315] – Recordings tab of the RE showing lots of [removed]
  • [MBS-6349] – Artist credits display wrong on add release edit
  • [MBS-6364] – "Possibly duplicate artists" report doesn’t show any results
  • [MBS-6374] – Release editor seeding is broken
  • [MBS-6383] – Can’t add search hint to area
  • [MBS-6416] – release editor loses track identifiers when editing a medium.
  • [MBS-6467] – "you may add a new label" in area search
  • [MBS-6471] – "Keep me logged in" is broken

Improvement

  • [MBS-2782] – "Edit medium" edit type > Don’t show pointless artists credits changes (e.g. when only join phrase has been changed)
  • [MBS-2848] – New "Add medium" artist credits section is cluttered and hard to parse
  • [MBS-3663] – "No" votes should allow time for the original editor to respond
  • [MBS-4652] – Add a link to How To under Documentation
  • [MBS-5911] – Display source for non-CAA cover art
  • [MBS-6205] – Clean up Soundtrack Collector URLs
  • [MBS-6207] – Check formatting for Soundtrack Collector URLs
  • [MBS-6259] – Bring back date/country columns in tables
  • [MBS-6352] – ExportAllTables::check_tables_completeness should be updated and used
  • [MBS-6363] – Update Twitter URL cleanup to use https

The Git tag for this release is v-2013-06-25.

Libdiscid 0.5.1 released

A new libdiscid version was uploaded today.

Changes:

  • LIB-40: discid_get_webservice_url() (web service version 1) is deprecated.
    Please use libmusicbrainz to gather metadata by disc ID
  • LIB-7: Rewrote data track handling, releases with multiple data tracks. This also fixes LIB-18 (no ID for DVDs) and LIB-9 (PS/PS2 CDs)
  • LIB-44: fix invalid disc IDs on first read of multi-session discs
  • LIB-37: Autotools optimization (non-recursive build etc.)
  • LIB-42: remove Windows 9x platform code
  • renamed openbsd platform code to netbsd, still used by both.

The data track/multi session disc handling was rewritten. This was started by Lukáš Lalinský quite some time ago and I finished this now. This should fix several issues with data tracks and data track handling works the same on all platforms now.

Christophe Fergeau contributed some changes to remove lots of clutter from the autotools build. This stops changing directories during the build all the time (non-recursive build) and should make it easier to see actual problems when building.

Additional contributions come from Philipp Wolfer and Sebastian Ramacher again.

Feedback needed:

We are (still) a bit stuck in the discussion for LIB-28 which also blocks several connected things in python-discid and possibly Picard.
The main question is how to name the “default device”, especially on Mac OS X. Usually real/internal device names are used. However, especially on Mac OS X these device names change quite frequently for various reasons, even when the same physical disc drive is “meant”. So I don’t know if using these makes any sense. It also might not be good to add these device names to a configuration file, since they will “break” easily (opening a .dmg before inserting the disc already breaks it).
I proposed using “1” to define “use the first disc drive”. I am no actual Mac user and really would like to know how you guys feel about this.
For Windows the drive letters are much more stable, but can change when USB disc drives are used. Using “1” is an option, but might make less sense, since drive letters are actually known to the normal user.
On Linux/BSD/Solaris the disc drive names are only used for disc drives (not for hard disks) and are quite stable.
This relates to python-discid#30 (should DEFAULT_DEVICE be a constant?) which again somewhat blocks PICARD-503 (using python-discid). This question is the only thing keeping python-discid from a stable API release.

LIB-28 is probably the best place for feedback, but you can also just answer here or in the announcement mail.

Information, documentation and other links are at:
http://musicbrainz.org/doc/libdiscid
That includes builds for Windows and Mac OS X.

Search server update: June 13

On 13th June we updated the search servers once more. Thanks for fixing bugs and adding Area support, Paul!

Release Notes – MusicBrainz Search Server – Version 2013-06-13

Bug

  • [SEARCH-297] – Webservice Json output for aliases when searching is inconsistent with output when doing a lookup
  • [SEARCH-302] – search server json output use singular for a list of release-groups.

Improvement

  • [SEARCH-292] – Include area info in the indexed search artist and label results
  • [SEARCH-299] – Ouput TrackIds

New Feature

  • [SEARCH-301] – Search for Area by ISO 3166 code

Task

  • [SEARCH-273] – Support for multiple country/release events on release as as part of schema changes
  • [SEARCH-286] – Add areas to the indexed search

Server Update, 2013-06-10

Another week, another release! This release is mostly bug fixes, but I’m sure a lot of people will be happy to notice that we have finally resolved MBS-357. That’s right – we no longer have a single clear text password in the MusicBrainz database. We’re sorry that it took so long, but at least it’s finally been done! Other than that, this release mostly builds on top of the new schema change features, and provides bug fixes for things that have regressed.

Many thanks to Michael Wiencek, Nicolás Tamargo and the MusicBrainz team for their work in this release. Here’s what’s changed:

Bug

  • [MBS-6007] – JSON release webservice doesn’t include works
  • [MBS-6179] – Sorting a collection by catalog number sorts ‘8BP117’, ‘8BP126’ and ‘8BP127’ in the wrong order
  • [MBS-6260] – Dates and countries are displayed inconsistently
  • [MBS-6263] – Adding an area without ISO 3166 codes shows blank values in edits
  • [MBS-6264] – Languages/Scripts statistics have an empty "Last updated field"
  • [MBS-6297] – Can’t create relationships from areas to other entities from the area page, but only from the other entities’ page
  • [MBS-6325] – Moving a release with a release date into an empty release group does not provide it with a date (sorting doesn’t work)
  • [MBS-6326] – ModBot is unable to close some edits as they still use ‘country_id’
  • [MBS-6334] – Release editor: loses join phrases and artist credits when switching to tracklist tab
  • [MBS-6355] – 502 displaying relationships for prolific artist
  • [MBS-6357] – Track/medium tables have indexes/constraints with bad names
  • [MBS-6370] – Beta: Inline search can load duplicate results
  • [MBS-6377] – Track ACs appear changed but the edit isn’t submitted
  • [MBS-6385] – Strange results when pasting an area URL in the inline search
  • [MBS-6409] – ModBot leaves translated edit notes
  • [MBS-6441] – Release (date, country) are blank when displaying a disc ID
  • [MBS-6443] – Ascending and descending sorting of releases in a collection results in the same order

Improvement

  • [MBS-357] – Don’t store passwords in clear text
  • [MBS-4962] – Slave servers should cache heavily due to hourly update process
  • [MBS-6203] – Only allow recognised URLs for relationship types which are for specific sites
  • [MBS-6233] – Ensure SecondHandSongs links are added at the right level
  • [MBS-6300] – Link to countries in country statistics
  • [MBS-6301] – Consider linking to areas {artist,release,label} list in country statistics
  • [MBS-6347] – Show parents somehow for underlying areas
  • [MBS-6358] – Add basic area stats display
  • [MBS-6362] – Improve how release data is displayed in Google searches
  • [MBS-6371] – Extend Wikipedia and Wikidata autoselect to areas

New Feature

  • [MBS-6269] – Expose track identifiers in the webservice
  • [MBS-6270] – Expose track identifiers on the website

The Git tag for this release is v-2013-06-10.

Server update, 2013-05-28

Now that the fires around the last schema upgrade seem to be dying down to a much more manageable level, we’ve just put out the next version of MusicBrainz server. As you might already expect, this release is mostly a bug fix release. Thanks to Lukáš Lalinský, Michael Wiencek, Nicolás Tamargo and the rest of the MusicBrainz team for their work on this release. Here’s what’s changed:

Bug

  • [MBS-3059] – Using "Submit votes & edit notes" added edit note to all edits
  • [MBS-6158] – Reset password form claims the user/email address does not exist
  • [MBS-6172] – Lost password claims to work when user has no email
  • [MBS-6209] – Direct search for work doesn’t check aliases
  • [MBS-6230] – Internal server error removing a relationship in the relationship editor
  • [MBS-6271] – Internal server error trying to set track lengths
  • [MBS-6274] – Capitalisation of "release events" in sidebar and edits is inconsistent
  • [MBS-6299] – "License" header in release sidebars is displayed even if the following list is empty
  • [MBS-6302] – Country column empty in search results page
  • [MBS-6304] – Table on Attach CD TOC page broken (too many columns)
  • [MBS-6305] – Regression: Release dates are no longer optional
  • [MBS-6318] – Search hint alias for areas require a sortname when adding
  • [MBS-6319] – Internal server error editing an area alias
  • [MBS-6329] – REGRESSION. no more dates in collection pages
  • [MBS-6335] – Regression: Tracklists not shown when merging releases
  • [MBS-6341] – Area alias aren’t found by direct search unless they match an area name

Improvement

  • [MBS-3083] – Do not use placeholder text for artist credit names
  • [MBS-5603] – Edit relationships page should show disambiguation comments
  • [MBS-6198] – Deleted Editor Stats
  • [MBS-6213] – Permit ISRCs starting with TC
  • [MBS-6219] – Stop using two edit types and two places in the UI for the "change release group" functionality
  • [MBS-6222] – Update FeaturingRecordings report
  • [MBS-6223] – Display language and script in the right order on release edits
  • [MBS-6294] – Release events are weirdly indented on release pages
  • [MBS-6340] – Redirect to direct search for indexed area searches as long as the latter are not actually available
  • [MBS-6348] – Display the type of area on inline search

New Feature

  • [MBS-6018] – Add /oauth2/tokeninfo and /oauth2/userinfo handlers for 3rd party login

Task

  • [MBS-6229] – Add autoselect, sidebar links and pretty URLs for Wikidata

The Git tag for this release is v-2013-05-28.

Search server regressions fixed

Yesterday we pushed out a new version of our search servers to fix some regressions introduced last week. Thanks to Paul Taylor for fixing these bugs so quickly.

Release Notes – MusicBrainz Search Server – Version 2013-20-05

Bug

  • [SEARCH-290] – REGRESSION WS2 RECORDING query returns cropped artist-credit
  • [SEARCH-294] – REGRESSION:Search results no longer include medium-list count attribute
  • [SEARCH-298] – REGRESSION:ws/1 release search seems broken

Improvement

  • [SEARCH-296] – Update README to point to up-to-date mmd-schema repository

Search server release: 2013-05-15

Coninciding with our main server release, we’ve updated our search servers. This version fixes some bugs from the last release and adds support for countries and track ids.

Thanks for your hard work on this release, Paul!

Release Notes – MusicBrainz Search Server – Version 2013-05-15

Bug

  • [SEARCH-236] – Incomplete VA artist credit included for releases in recording search
  • [SEARCH-282] – REGRESSION:Johanne Sebastian Bach is not the first result when search for artist Bach
  • [SEARCH-283] – REGRESSION:"-" is returned instead of an empty list when there are no ISWCs for a work
  • [SEARCH-284] – REGRESSION:"-" is returned instead of an empty list when there are no ISRCs for a recording

Improvement

  • [SEARCH-219] – Include alias sortnames when searching labels or artists
  • [SEARCH-257] – entity search : entity name should have more weight than aliases and artist credits
  • [SEARCH-268] – Add extended alias info to the ws search results
  • [SEARCH-269] – WS searches don’t return aliases that match the artist name

Task

  • [SEARCH-274] – Support for changes to Countries in forthcoming Schema release
  • [SEARCH-285] – Support for TrackIds in forthcoming Schema Release

Schema change release, 2013-05-15

Today we released a schema change update for MusicBrainz. Schema change updates change the format of the underlying MusicBrainz database and allow us to store more information, or model information in a richer/more correct form.

Summary

This release specifically includes some exciting new features:

Areas

A new entity is this release is the “area” entity, which can track countries, subdivisions of countries, cities, and other such location entities (venues, however, will be another entity). While this release primarily only introduces the entity, migrating only our existing list of countries, it’s now also possible to add start and end locations to artists, and to mark works as anthems. Editing of areas, their aliases and annotations, and area-area and area-url relationships, is limited to a new class of “Location Editor”.

ISNI Codes

The International Standard Name Identifier (ISNI) is an ISO standard that identifies public identities of parties. We now have support for storing ISNI codes inside the MusicBrainz database, which will make it easier to cross-reference data in MusicBrainz with other databases.

Multiple Release Events

Releases can now have multiple date and country pairs, whereas previously they could only have one country and one date. This will allow us to more accurately store information about releases that occur in different areas at different dates, but are otherwise the same physical product.

Forthcoming Features

We also began work on the database support for some future MusicBrainz features:

Track MBIDs

All tracks on mediums now have unique identifiers. This will allow people to refer to a specific track in a release in a way that is more resilient to editing than just the track name or position. Currently we have database support for this, but track identifiers are not yet exposed in either the website or the web service.

Dynamic Work Attributes

Dynamic work attributes will let us introduce new attributes to describe works without schema changes

Free Text Relationship Attribute Credits

This feature will let editors specify an alternative name for relationship attributes to specifically exactly which model guitar was used in a recording, rather than the current vague “electric guitar” attribute. Support for this feature is now in the database (in the link_attribute_credit) table, but the UI to do this editing is still to be finished.

Support more formats in the Cover Art Archive

Uploading cover art to the cover art archive will soon support a few other image formats, starting with PNG.

Other Schema Changes

The remaining smaller schema changes are:

  • The wiki transclusion version mapping is now stored in the database, not a flat file.
  • The link_type.short_link_phrase was renamed to link_type.long_link_phrase.
  • The work.artist_credit column was dropped.
  • Collections can now have a description.

Upgrading

If you are currently running a slave database, then you will need to perform a few manual steps to upgrade to the new version:

  1. Take down the web server running MusicBrainz, if you’re running a web server.
  2. Turn off cron jobs if you are automatically updating the database via cron jobs.
  3. Make sure your REPLICATION_TYPE setting is RT_SLAVE
  4. Switch to the new code with git fetch origin followed by git checkout v-2013-05-15
  5. Run carton install --deployment to install any new perl modules.
  6. Run carton exec -Ilib -- ./upgrade.sh from the top of the source directory.
  7. Set DB_SCHEMA_SEQUENCE to 17 in lib/DBDefs.pm
  8. Turn cron jobs back on, if needed.
  9. Restart the MusicBrainz web server, if needed.

Please see the instructions in our more recent blog post instead of these instructions.

Release Notes

This release wouldn’t have possible without help from Alastair Porter, Michael Wiencek, Nicolás Tamargo or the rest of the MusicBrainz team – thank you all for your hard work! As we missed the previous release, there are a few other changes in this release. Here are the full release notes:

Bug

  • [MBS-4703] – Add Medium edit does not correctly display the auto-edit note
  • [MBS-5834] – Users’ votes page mistitled as “edits”
  • [MBS-5851] – Instruments are missing from the JSON webservice responses
  • [MBS-5979] – Automatic redirect to beta clear release editor seeding
  • [MBS-6015] – Edit Artist Credit edits affecting track ACs don’t appear in related release edit histories
  • [MBS-6129] – Relationship editor isn’t correctly parsing attributes
  • [MBS-6149] – Entity merges silently dropping aliases with locales
  • [MBS-6178] – URL page headers are completely inconsistent
  • [MBS-6196] – work/edit_form.tt includes artist credit docs
  • [MBS-6249] – Add Event button broken on add release with cdtoc

Improvement

  • [MBS-1346] – New Report: Artists with 0 subscribers
  • [MBS-2229] – Allow multiple release events per release
  • [MBS-3626] – Display license logos in the sidebar
  • [MBS-3669] – Merge dated and undated relationships
  • [MBS-4115] – Cover art archive: Support .png SQL changes
  • [MBS-4294] – Add a “description” field to collections (SQL/UI)
  • [MBS-4756] – Move the wiki transclusion index to the database
  • [MBS-4866] – URL autoselect: ameblo.jp -> has blog at
  • [MBS-4867] – Timeline graph rate-of-change graph should change its vertical scaling depending on where it’s zoomed
  • [MBS-4925] – Add country of birth and country of death to Artist (person)
  • [MBS-5528] – Change short_link_phrase to long_link_phrase
  • [MBS-5772] – Generate relationship documentation (semi-)automatically
  • [MBS-5848] – Instrument credits (SQL)
  • [MBS-6023] – Track MBID UI changes
  • [MBS-6141] – Add discography page URL matching for universal-music.co.jp, lantis.jp, jvcmusic.co.jp, wmg.jp, avexnet.jp and kingrecords.co.jp
  • [MBS-6142] – Prevent Wikipedia links from being added as discography page relationships
  • [MBS-6188] – Remove rating from work merge page
  • [MBS-6189] – Show work languages on ISWC page
  • [MBS-6190] – Artist credit diffs per word, join phrase per char

New Feature

  • [MBS-799] – Location, venue and event support
  • [MBS-1839] – Track MBID SQL changes
  • [MBS-2417] – Support multiple countries/regions on a single release
  • [MBS-3296] – Add dynamic attributes
  • [MBS-3985] – Support multiple artist countries
  • [MBS-5272] – Create daily and weekly “rollup” replication packets
  • [MBS-5302] – Store International Standard Name Identifier (ISNI, ISO 27729) for artists and labels
  • [MBS-5861] – Dynamic work attributes (SQL)

Task

  • [MBS-5314] – Drop the work.artist_credit column
  • [MBS-6133] – Ensure JPEG uploads still work through CAA
  • [MBS-6167] – Add iTunes links to the sidebar
  • [MBS-6170] – Add Bandcamp links to the sidebar

Sub-task

  • [MBS-4217] – Spotify relationship under the External links section
  • [MBS-5809] – SQL changes for MBS-4294: Add a “description” field to collections
  • [MBS-5917] – transclusion to DB: SQL
  • [MBS-5918] – transclusion to DB: UI
  • [MBS-5919] – locations: SQL
  • [MBS-5920] – locations: UI
  • [MBS-5929] – Schema changes to store relationship type guidelines and example usages in the database
  • [MBS-5930] – Generate documentation automatically/allow managing guidelines in DB
  • [MBS-5933] – Schema changes to support multiple (date, country) pairs on releases
  • [MBS-6187] – UI changes to support multiple country/date pairs on releases

libdiscid 0.5.0 released

A new libdiscid release was made available today.

Changes:

LIB-29: add read_sparse() for faster reading again
LIB-35: add HAVE_SPARSE_READ and VERSION_* defines
LIB-36: hide internal symbols on Linux/Unix
LIB-34: distmac and distwin32 cmake targets

The important change is the read_sparse() function:

Philipp Wolfer added a read_sparse() function. With this function you can either only read the TOC or specifiy which features of the disc you want to read.
The normal read() also extracts ISRCs starting with 0.3.0.
You might want to change existing applications to use read_sparse if you care about performance and don’t use ISRCs. The TOC is usually cached, so read_sparse() can be faster (0,5 vs. 3 seconds measured).
The difference is only in where the time is spent. It doesn’t really save overall time, but the TOC is read right when the disc is inserted so no additional disc access is performed when using the TOC in your application.
To make it possible to keep using read() when read_sparse() is not available we provide the HAVE_SPARSE_READ define, which can be used like that:

#ifndef DISCID_HAVE_SPARSE_READ
#define discid_read_sparse(disc, dev, i) discid_read(disc, dev)
#endif

discid_read_sparse(disc, device, 0)

We also provide defines for the libdiscid version numbers.
However, you should rather test for features/functions in the build files and create specific defines for your use case.
The above define is only provided as a convenience for read_sparse().

There are more details about the other changes in the full announcement mail.

If you didn’t follow the musicbrainz-devel list:
This year brought several new releases for libdiscid, starting with ISRC and MCN support in libdiscid 0.3.0. Applications using libdiscid 0.2.2 (or even lower) still work with libdiscid 0.5.0.

Information, documentation and other links are at:
http://musicbrainz.org/doc/libdiscid
That includes builds for Windows and Mac OS X.

Server Update, 2013-04-08

We’ve just finished pushing out another fortnight of work to the main MusicBrainz servers. While work continues on the forthcoming schema change release, we have fixed a few bugs and added a few improvements. Thanks to Lukáš Lalinský and the rest of the MusicBrainz team for their work on this release! Here’s what’s changed:

Bug

  • [MBS-5524] – Edit relationship type edits are missing information for identifying the relationship type
  • [MBS-5830] – Attributes are not shown in relationship type edits
  • [MBS-5964] – Artist credit’s sort name tooltip doesn’t update after edit
  • [MBS-5980] – Cover art comments can make the cover art page look very unstructured
  • [MBS-5993] – name variation checks happen after HTML entity conversion
  • [MBS-6019] – Some FreeDB data getting wrongly imported from the dumps
  • [MBS-6021] – Track count search in Add Disc uses full release count for multi-disc releases
  • [MBS-6038] – "See all {num} {entity}" is just wrong i18n-wise
  • [MBS-6042] – Release country field in edit search breaks after doing a search
  • [MBS-6043] – In front page "Recent Additions" the text for the cover art pop-up labels (HTML title attribute) is wronlgy escaped
  • [MBS-6075] – An error occured while loading this edit

Improvement

  • [MBS-5613] – Set cover art page does not show enough information
  • [MBS-6039] – Make it possible to display attributes after the link phrase

Sub-task

  • [MBS-4258] – Make dates translatable

The Git tag for this release is v-2013-04-08.