Month: January 2013

IMPORTANT: Proposed changes to the data returned by our web service

Our current web service at the /ws/2 endpoint returns too much data in a lot of cases and in many cases we suspect that the programs making the calls to the service don’t actually consume all of that data. We’d like to reduce the amount of unused data our web service returns, in order to reduce our bandwidth costs. We propose that:

  • The web service will no longer includes aliases and tags in relation elements. Regardless of what entity you may request, if the results of your request includes a relation element, any alias or tag elements that are currently returned will no longer be returned.
  • The web service no longer includes aliases and tags in for the Various Artists artist anywhere, unless you specifically request the Various Artist from the /ws/2/artist endpoint.

We’ve mocked up these changes in the following XML files:

We think that this will have a minimal impact on our web service users. If you use our web service, please tell us what you think about this. If you know someone who is using our web service, but may not read this blog, please forward a link to this post to them.

For more background on our research into this topic, please take a look at this document.

Server Update, 2012-01-28

We’ve just finished releasing the last release for January, 2013. This release is a combination of bug fixes and improvements to existing features, and was worked on by Frederik “Freso” S. Olesen, Michael Wiencek, Nicolás Tamargo, Pavan Chander and the rest of the MusicBrainz team. Thanks for your hard work, everyone! Here’s what’s changed:

Bug

  • [MBS-5022] – Release Editor: Missing entities page doesn’t handle 0 or only 1 result
  • [MBS-5453] – Bad redirect(s) https→http
  • [MBS-5540] – Release editor eats tracklists
  • [MBS-5586] – ISE when trying to submit an “Edit artist” edit with no changes
  • [MBS-5609] – JSON webservice doesn’t send relationship attributes
  • [MBS-5614] – Links to edit notes to other edit notes always use http
  • [MBS-5691] – Artists’ recording lists show ISRCs but don’t link to them
  • [MBS-5699] – CD stubs use “disambiguation” instead of “comment”
  • [MBS-5711] – IPIs missing from IPI element in JSON webservice
  • [MBS-5712] – JSON webservice handles missing values inconsistently
  • [MBS-5713] – Alias types missing from the JSON webservice

Improvement

  • [MBS-4536] – Include the presence of cover art in the webservice
  • [MBS-5304] – “Other Databases” whitelist: “Trove” ammendment
  • [MBS-5447] – Bandwidth performance: revert MBS-4764 now that thumbnail quality has improved
  • [MBS-5606] – Implement STYLE-160 (The Session) in code
  • [MBS-5651] – Clicking link for packaging types loses all entered information
  • [MBS-5652] – Release Label sorting makes no sense on Release Group page
  • [MBS-5702] – Change the text on the “Read more…” link under wikipedia extracts
  • [MBS-5739] – Disconnect database handle on page timeout
  • [MBS-5741] – Time out edit searches in 45 seconds, rather than 30.
  • [MBS-5749] – 401 error: Add link to profile edit page

Task

  • [MBS-5758] – Remove schwag/store links from musicbrainz.org pages

The Git tag for this release is v-2013-01-28.

Privacy policy inconsistencies

Recently we’ve received two bug reports that point out two inconsistencies in our privacy policy:

  • MBS-5708: It’s not possible to disable the display of cover art but the privacy policy claims it is. There are two possible options for fixing this; fix the privacy policy or make a new preference. Which would you prefer?
  • MBS-5709: Inclusion of Google Analytics is in violation of the privacy policy. This one is more tricky, since we link to other third parties (archive.org, gravatar, captcha) that are also not mentioned in the policy. And changing the policy for each time we add a new third party becomes cumbersome. No clear solutions have formed around this issue, so we would like your feedback on this.

If you care about our privacy policy, please take a moment to read these bugs and comment on them. Thanks!

7digital & The Echo Nest have become MusicBrainz customers

I’m pleased to announce that 7digital and The Echo Nest have become our latest customers!

7digital enables a lot of digital music stores and provides a lot of services for mobile operators. 7digital has relationships with many labels and thus faces complex metadata issues. I’m quite pleased that 7digital has chosen to partner with MusicBrainz to fix these metadata issues.

The Echo Nest provides tons of digital music services and is a driving force behind Music Hack Days here in the States. The Echo Nest also created project Rosetta Stone, a service that translates to/from MusicBrainz IDs from/to other ID spaces like the Echo Nest IDs or Rdio IDs.

Welcome to the MusicBrainz ecosystem!

All Our Git Repositories Are Moving To GitHub

On February 11th 2013, we will be shutting down the MusicBrainz hosted Git repository, in favour of using GitHub for hosting. While we will continue to take backups, these will be internal and thus git.musicbrainz.org will cease to operate as it currently does.

There are a few reasons for this move, but the predominent reason is so MusicBrainz can host less. We currently host a lot of our own software and this is straining the already-limited resources we have. If someone else can do something just as well or better than we can – then let them! Using GitHub also means we get a slightly better repository browser, which can make communicating and discussing code easier. However, we are not moving to pull requests, though we may in the future.

We already mirror git.musicbrainz.org onto GitHub, so if you are only interested in read access, we recommend changing your local clone now, to save confusion in the future. To do this:

  1. cd to your musicbrainz-server clone.
  2. Run git remote set-url origin git://github.com/metabrainz/musicbrainz-server.git

That’s it! You should now be able to git pull just as you did before.

GitHub has a few other possible protocols for cloning, so if HTTP or HTTPS is better for you, then check out the GitHub repository for all the possible links.

If you’re a developer currently developing against our repositories, then you should either fork the repository on GitHub, or contact us to talk about getting your keys added.

Sorry for the disruption!

Server update, 2013-01-14

After delaying this release by a week due to Christmas/New Year getting in the way, we have pushed out a new update of the MusicBrainz web site. This release consists mostly of bug fixes and a few outstanding tasks that we still needed to complete. Thanks to Frederik “Freso” S. Olesen, Michael Wiencek, Johannes Weißl and the MusicBrainz team for their work on this release! Here’s what’s changed:

Bug

  • [MBS-5288] – Use API when checking for WikiDoc changes
  • [MBS-5577] – Relationship Editor displays "vocal vocals"
  • [MBS-5663] – Leap year in "Edit Relationships"
  • [MBS-5716] – Reorder mediums edit is missing a space between medium title and old position
  • [MBS-5717] – Reorder mediums edit for mediums without titles shouldn’t include colons

Improvement

  • [MBS-5723] – Allow sorting collections by all columns, not just a select few.

Task

  • [MBS-3856] – Revert MBS-1052 "Amazon should lookup cover art by barcode too"
  • [MBS-5729] – Consolidate Metabrainz:Sponsors and About/Sponsors

Sub-task

  • [MBS-3974] – Filter by release country

The Git tag for this release is v-2013-01-14.

A sad day for the Internet: RIP Aaron Swartz

As you’ve probably seen around the net today, Aaron Swartz, Internet Hero has committed suicide.

Aaron Swartz has spent most of life working to improve the Internet and to preserve freedom on the net. Many people are speaking to his awesome accomplishments in the last 10 years of his life, but I’d like to take a minute and reflect on his earlier years.

I was one of the fortunate people who met Aaron when he was still 15 — we first met up in Washington DC for O’Reilly’s P2P conference. Since he was a minor, his mom was accompanying him. Never mind that she had a broken leg at the time — she was so dedicated to her son that she traveled with him to allow him to participate in things that most minors couldn’t even imagine.

Before I met him in person, Aaron was an active contributor to MusicBrainz. When I started my first mis-guided attempts to create an RDF based web-service, he worked with me to improve the schema. He helped me understand RDF (damn that RDF spec!) and helped me fix the schema until it actually worked properly. Aaron was always looking for new and interesting things to do, so once his mission with MusicBrainz was done, he moved on to bigger and better things. And the things he did — simply amazing that one person can accomplish so much in so little time.

Aaron and I shared one passion — making data open and accessible. His means were always more aggressive than mine; he often chose the faster, more risky approach. I usually favor the slow-and-steady-will-win approach. Regardless, the events that led up to his suicide leave me deeply unsettled about the current state of affairs.

Aaron, thank you for being the instigator, shit-stirrer, advocate and dissident you were. I appreciate everything you’ve done during your short stay here in this troubled planet. May your next journey be more peaceful!

Thank you to Cory, Larry and Brewster for your kind words.

UPDATE: Here is a link to the paper Aaron wrote about MusicBrainz.

Replication issues, and packet 64833 is large

  1. We’ve ushered in the new year by discovering, then solving some issues with replication; packet number 64831 (from yesterday, 1AM UTC) didn’t build correctly and needed a bit of manual prodding. However, it’s now been pushed out and replication should be back to normal.
  2. As part of our fix process, we turned off production of replication packets for the bulk of today. As a result, packet number 64833 covers what would otherwise have been about 20 packets, and thus is somewhat large. The import process for this packet will accordingly take somewhat longer than usual.

Sorry for any inconvenience, and happy new year!