MetaBrainz Blog

We’ve gotten a slew of bug reports overnight as to what was wrong with the latest release. Keschte and I worked hard to address a number of these issues and I’ve done a mini update on the server. Check out this list of recently closed bugs to see what we did. Thanks to all those who reported bugs!

The broken tagger issues should also be resolved now.

Technorati Tags: release, webserver

We just updated the main server with the long awaited new release:

Not really new, or not so obvious features

• The MusicBrainz web pages have been polished and cleaned to be web standards compliant. We proudly serve the pages in XHTML 1.1, CSS 2.0 compliant mode.
• The user interface has been given a face lift and a cleaner look.
• We have made great steps on the terminology changes, which had hung in limbo for quite some time. From now on, the term ”album” is reserved for release attributes. The albums are now called ”releases”. The same is true for the term ”moderation”: we replaced this with the more appropriate term ”edit”.
• Lots of static pages have been replaced with WikiDocs pages. This means that most of the documentation pages displayed on this very server are no longer static html pages, but are instead content transcluded from the MusicBrainz Wiki. For each of these pages, a given revision is labeled as “official content”, and is displayed inside the MusicBrainz site. (For example, compare http://musicbrainz.org/doc/ContactUs to http://wiki.musicbrainz.org/ContactUs). If the content is not labelled as official, it can still be browsed in the same way, but will be labelled as unofficial content. WikiDocs is a huge project, and some parts will always be in the works. Please talk to the Wikizens if you’d like to contribute!

New features

• The “Release Editor” (nicknamed “The edit page to rule them all”) replaces the FreeDB Import, the CD-Lookup as well as the “Edit all” workflows. It supports all the edit types usually applied to core entities (artist/release/tracks), including single artist conversions, various artist conversions, but also explicit changes to one track artist of a single artist release. This will allow us to more flexibly edit releases to conform to the new ReleaseArtistStyle, all in one go.
• Timestamps on edit notes.
• Diff display of changed titles on the voting (previously: moderation) pages.
• Edits are linked to their respective documentation.
• Edits show all the parent objects that are relevant for the current object (example: on a track title edit, the artist and the release are shown in the header).
• Pending edits are shown and explained in more detail on the edit pages. Links are provided to review the pending edits.
• The help blurbs on the editing pages were extended, and now include links to official WikiDocs documentation relevant to the current EditType.
• GuessCase classical mode.
• The Indexed Search was extended to include additional fields for the Artist Search. Users can choose to include artist names, sort names and alias’ for more precise search results.
• The Indexed Search results have an additional row which allows to add the entity to the Create Relationships list. This should allow to enter relationships much faster now.

Improved features

• Lots of fixes for the GuessCase function.
• Numerous others: please see the closed tickets for the current Milestone and Version.

Many thanks to Keschte and everyone else who contributed and tested for your hard work on this release!

Technorati Tags: musicbrainz, release, webserver

There are two web servers running on the main web server machine. The first web server is light and handles all the content that is simple, such as static pages, images and the like. Anything that requires more intelligence, such as talking to the DB, gets passed to the second web server, which is designated for these heavy requests.

The light server will wait for a specified time for the heavy server to finish its job — currently 120 seconds. If the heavy server hasn’t finished the job in that time, the light server gives up and and returns you the dreaded 502 error. The DB server will unfortunately continue to chug on the query and finish executing it as requested — cancelling an existing query is hard to do, and often times its better to let the server just run its course.

The gut reaction might be to say: “Why not stick around longer and wait for the results, if the DB is going to crank them out anyway?” Problem is that if we do this, the light web server is sitting idle doing nothing while waiting for the DB/heavy server to finish its job. The light server can give up and instead spend its time better doing things it can accomplish in a reasonable amount of time — like serving smaller requests for others. With this setup, the overall system favors the less intensive requests and thereby increasing the overall number of queries that were successfully handled. If we stopped and waited for the DB/heavy server to finish its stuff, we would pretty quickly clog up the web server with requests that are sitting idle, doing nothing. And that clog would then prevent any further connections to the web server and the whole site comes to a halt.

If you want a visual representation of what is going on, check the load graphs for dexter, our DB server. Any load greater than 4.0 and the DB server is no longer running optimally. We’re fine right this second, but in 10 minutes time?

So, what are we doing about this?

1. Optimize the server code so that the user cannot make these intensive requests
2. Spread the DB load across multiple replicated slave servers.
3. Partition the database so that we can have multiple master servers. For instance, we could have one DB server that handles all the edits and one that handles the data. Maybe one that handles TRMs and PUIDs. This way each machine does less work, but this is a lot of work to code for mb-server devels.
4. Find someone to give us a beefy database server with 12GB – 16GB of RAM

So, next time you’re aching for new mb-server features, please keep in mind that we’re spending a lot of time just keeping the service running smooth. Our income isn’t great enough yet that we can hire people to maintain the site AND hire people to hack on new features. In the meantime, Dave Evans and I will focus on keeping things running and hard working folks like Keschte are working on new features for the server. Overall we’re still moving forward, just a lot slower than we care for.

What can you do?

1. Help us solve our DB issues if you’re a DB person.
2. Help us write more mb-server code.
3. Most important of all, make a donation!!
4. Bug your rich friends to donate to MusicBrainz so we can buy a beefy database server. 🙂

Technorati Tags: hardware, hosting, webserver