Upcoming releases: Release groups and Next Generation Schema

In the past month there has been a ton of activity behind the scenes here at MusicBrainz and I can finally give a cohesive update on our plans for the next few months.

The much anticipated Release Groups release has been coded by Lukas in a weekend code sprint based on the old Mason codebase. Even though we had declared the old codebase as end-of-life, we have decided to push release groups out using the older code in order to satisfy the needs of the BBC and other customers. As part of this release, I will also add ISRC support and include a handful of bug fixes. Expect this release in May — I’ll post again when the date is firmly set.

And what is even more exciting is that we’re about to start work on our much anticipated Next Generation Schema (NGS). Discussed and planned over and over again, we’ve finally settled on an approach that appears to make everyone happy. As part of SoC, we’re likely going to accept Oliver and Lukas’ proposals to work on NGS over the summer. The goal is to implement all of the new schema in one release based on the TemplateToolkit/Catalyst work that Oliver has been working on since last summer. We’re going to take a step back and create a new object model/schema and then glue the TT UI onto the new object model.

The schedule puts the TT/Catalyst/NGS release into final beta test on August 31, with a release following in 15-30 days after that date. Please note, however, that there will be no other release based on Oliver’s TT work before NGS is release in September. We had to skip that release in order to pull in the schedule to make the target date of August 31.

I am quite excited by the work that is being done in the server area right now — we’re on our way to get past some significant hurdles. Just yesterday I got a first glimpse at the MusicBrainz site partially translated to Dutch — startling at first, but quite exciting when you think about it.

Many thanks to Matt Wood at the BBC for having the patience and dedication to work with MusicBrainz. Many thanks to Lukas for the coding sprint to get Release Groups off our collective plate. And of course many thanks to Oliver, Nikolai, Brian for your continuing hard work on TT. And thanks to everyone who has been supporting this team over the past few months.

Wiki Migration

Today’s the day – our wiki is being migrated to MediaWiki.  The old “moin” wiki is now read-only (and will remain so, at least for a few months), and is available on oldwiki.musicbrainz.org.  The new wiki, once all the data has been migrated across, will be at the usual address.

As soon as the migration is complete, I’ll switch wiki.musicbrainz.org over to point to MediaWiki.

Unfortunately it won’t be possible to also migrate the user accounts from moin to mediawiki, so regrettably this means that once mediawiki us up, you’ll have to re-create your accounts.  Sorry about that.

Update: the switch has been made – if you have any questions to ask or problems to report about this, please see the WikiMigration page.  Thanks!

MusicBrainz Flickr machine tags

Sander van Zoest and Dan Brickley have been prodding me to officially post about Flickr Machine tags — using Flickr machine tags you can now tag your photos on Flickr to refer to specific MusicBrainz entities. This is how to create MusicBrainz machine tags on Flickr:

  • musicbrainz:artist=<MBID>
  • musicbrainz:release=<MBID>
  • musicbrainz:track=<MBID>
  • musicbrainz:label=<MBID>

For more details see the official wiki page: FlickrMachineTag

Wanted: Wiki, Documentation, Trac and UserVoice guardians

In working on figuring out how to integrate UserVoice into our workflow its become painfully clear that we need to have some people take “ownership” over a few portions of MusicBrainz. Back in the day when Cristoph König (Don Redman) was our Wiki warden, our wiki was well groomed and worked smoothly. Alas Cristoph has been swallowed whole by a University in Germany and we may never see him again. 🙁

Its time to find volunteers who are interested in taking on personal ownership of these aspects of MusicBrainz. I’m looking for 2-4 volunteers, one for each of the following areas described below. However, please note that these people are not expected to carry out the bulk of the work that needs to be done to get these projects into an improved state. I envision each of these people being motivators and leaders who can focus and encourage the efforts of other volunteers to help them achieve their goals — there is a big difference between a leader and a workhorse.

  1. Wiki: This person should be well versed in Wikis and understand their general nature. A “Wiki Warden” should be willing to help with the upcoming wiki migration to MediaWiki and have a vision for how our wiki should be organized and cleaned up. This person should be willing to do more work initiallycleaning up the wiki and then spend a few hours every week watching over activity in the wiki and gently steer the wiki into a direction of sense and overall usefulness.
  2. Documentation: This person should be working with the Wiki Warden (it could even be the same person) to coordinate the creation/maintanance of Wiki pages for use with our WikiDocs documentation system. This person should be have an overall vision for how to organize our documentation and to move from our current state to a more organized and useful documentation system for MusicBrainz.
  3. Trac: This person should be familiar with trac and work to remove the cruft that has accumulated in the bug tracker. We need someone to close old bugs that no longer apply, highlight bugs that are important and generally reduce the duplication present in the system. It would be best if this person could also take part of the weekly developer chats in order to stay in tune with the development process.
  4. UserVoice: While a lot of people have volunteered to take part in working with UserVoice, we need one person to take the lead and be in charge of the process. I would love it if this person could help us to settle on one of the proposed UserVoice workflows.

I’d like to stress once again that I am not looking for people to do a lot of work — I’m looking for leaders who can do a little work, but motivate others a lot of help out and create a thriving sub-communities that allow MusicBrainz to become more concise and cohesive. If you’re interested in one or more of these positions, please post to the comments.

Problem delivering mail to gmail / googlemail

This week MusicBrainz experienced problems while trying to deliver mail to gmail.com / googlemail.com. The problem is now fixed, but regrettably this means that some messages that MusicBrainz should have sent are now lost.

This week MusicBrainz experienced problems while trying to deliver mail to gmail.com / googlemail.com. The problems started on Tuesday morning (UK time).  On Friday morning the problem was identified as a broken DNS server, which was then fixed, thus resolving the problem.

Regrettably this means that some messages that MusicBrainz should have sent are now lost.  The number of lost messages is approximately:

  • 103 ‘subscriptions’ messages from Tuesday
  • 61 ‘subscriptions’ messages from Friday
  • 297 other messages (new user signup, edit notes, etc)

Please accept our apologies for this error.

MusicBrainz Server Roadmap

After considering all of the options and taking in tons of feedback from our developers, our community and our customers, I’ve finally settled on the following road map for the MusicBrainz server. The plan allows us to follow the release early, release often methodology and should hopefully make most people happy:

TemplateToolkit/Catalyst Release

  • Date due: Late March/April
  • Non schema change release
  • Deployed as beta.musicbrainz.org starting in early march
  • 100% features of 2008-11-23 release supported.
  • Full Guess Case supported, but no new features
  • Many UI improvements, including a generally speedier interface
  • Internationalization support: Support will be under the hood, but we may not immediately support new languages.
  • Ease of installing mb_server: Use modern tools to draw in more server developers.
  • Search improvements: CD Stubs. Possibly: Advanced Relationships, edits
  • Closing a whole host of old bugs, introducing many new ones. 🙂

This release doesn’t add many new features, but in general we feel that the user interface experience will improve so much that our end users will be happy with this new release. Also, since we’re keeping to a non-schema change release, we will be able to load this release and let people test with live data on beta.musicbrainz.org. I believe this well get more people to come help us test the new interface and hopefully have a smooth rollout of the new features.

NGS Release

  • Date due: To be determined — hopefully sometime this summer
  • Major schema change release
  • Full NGS object model under the hood.
  • Expose ReleaseGroups, if not other new NGS features
  • Keep old edit system in place

The major piece of work in this release is the new NGS schema/object model. How we graft the existing edit system on top of the new schema remains to be seen — this task contains many unknowns, which is why it was yanked from the upcoming release. We’ll end up doing more work overall by having the TT/Catalyst release first, but this approach allows us to release early, release often.

NGS Improvements releases (may be one or more releases)

  • Date due: weeks after NGS
  • Non-schema change releases
  • Bug fixes
  • Incremental improvements in NGS
  • Exposing new NGS features if the NGS release didn’t expose 100% of the new features

Depending on how we work out the NGS release, I can see smaller follow up releases that improve the NGS release or expose more portions of NGS that weren’t previously exposed. Exactly how this shapes up will not become clear until we near the NGS release.

Edit System Rewrite

  • Timeframe: To be determined
  • Schema change release
  • Improve may aspects of our edit system.

Our edit system has been straining under its current load for some time. This release will throw out the edit system entirely and build a new more flexible system that allows the user to change more data with fewer edits and allows the user to find edits easier.

If you have questions on the roadmap, please post them in the comments.

Improving the MusicBrainz user experience: Would you like to help?

A few people have been commenting on how MusicBrainz has a few bugs that bother them on a daily basis, yet these bugs are never fixed. From a developers perspective its really hard to see which bugs in our large bug list really matter to end users — its hard to figure out which bugs should be fixed first. Rather than fixing the bugs that affect most people first, developers tend to fix the bugs that have the most insistent users shouting for those fixes. Unfortunately fixing bugs for overly vocal users may not be the same as fixing bugs that will improve life for the most people.

Warp suggested that we check out Get Satisfaction, while others suggested using UserVoice. Both of these services provide an intermediate layer between the developers and the end users to provide feedback about the bugs that are important to them. End users can enter the bugs that bother them and everyone can vote on those bugs/issues. While we already have a bug tracking system, this system is geared to the less technical users out there — not everyone loves entering bugs into trac. Plus UserVoice’s voting mechanism allows the most important bugs to rise to the top of the list.

I’ve explored UserVoice a little since they offer a free plan for Open Source projects. (Not to mention that their site is in Orange, which is a big draw for me. 🙂 ) It seems that it would be easy to integrate this with MusicBrainz. But, it seems that this is one more chore for someone to manage and the last thing I need is to take on another chore. I’m willing to do the technical integration, but I would need some help from volunteers to manage the day-to-day operations of UserVoice. I envision volunteers to pay attention to the data that collects at UserVoice and maintain a mapping between UserVoice issues and actual bug/enhancement tickets inside the MusicBrainz bug tracker. If you already play with trac and help us maintain it, this might be another aspect in which you could help MusicBrainz.

I’d like to know:

  • Is using UserVoice a good idea?
  • Do you see any potential problems in using it?
  • Would you be interested in being an Admin on UserVoice to help manage MusicBrainz’ UserVoice site?

Please let me know in the comments!