Failed drive in database server RAID array (updated)

Yesterday afternoon a drive failed out of our RAID array in the master database server. We have four drives in this server, with two sets of two drives (using a RAID-1,0 configuration) so a single drive failure isn’t a big deal. We’re still up and running, but we’re quite a bit more vulnerable until we … Continue reading “Failed drive in database server RAID array (updated)”

Yesterday afternoon a drive failed out of our RAID array in the master database server. We have four drives in this server, with two sets of two drives (using a RAID-1,0 configuration) so a single drive failure isn’t a big deal. We’re still up and running, but we’re quite a bit more vulnerable until we get this drive replaced. If the last drive in our RAID-1 array fails we’d have to deal with data loss for today — fortunately the data dumps ran last night and backed us up until then. Let’s keep our fingers crossed that we do not have any failures today.

Once I got notified that the drive failed, I promptly called Sun for a warranty replacement. In the back of my mind I thought: “I should order a new drive from NewEgg, just in case”. But, the Sun representative assured my that they would have a replacement drive for me today. This morning a clueless service engineer called me to tell me that they shipped the wrong drive to our data center. When I called back, they informed me that they did not have the correct drive and that the vendor would ship one to me by Thursday morning.

Having lost faith in Sun to do the right thing, I ordered a spare drive for the server from NewEgg — I should’ve done this a couple of months ago when the prices for these drives become reasonable. Now we’ll have a spare on hand for the next time this happens (and it will quite soon, I’m sure — both Seagate drives that originally came with the server failed within one year. The first drive failed within the first month!)

So, I am curious to see who will win: Sun who had an 18 hour head start to replace the drive, or NewEgg/UPS who got my order just this morning. I’ll post an update once this plays out — in the meantime keep your fingers crossed for no further drive failures!

UPDATE: DHL delivered the Sun drives 1 hour before UPS delivered the NewEgg drives. The drive is now in the server and the RAID array is working optimally again.

Unplanned downtime

The primary web server has just gone offline unexpectedly – we’re looking into it. Meanwhile you can still the other parts of MusicBrainz (read on for details). Sorry for the inconvenience – please bear with us, and check back here for updates. Thanks!

The primary web server has just gone offline unexpectedly – we’re looking into it. Meanwhile you can still the other parts of MusicBrainz (read on for details).

Sorry for the inconvenience – please bear with us, and check back here for updates. Thanks!

Continue reading “Unplanned downtime”

Happy new year: MetaBrainz reaches sustainability!

Now that the holidays are over its time to get back to work and kick MusicBrainz work back into high gear! Its been my goal for many moons to make MetaBrainz self sufficient in order to not have to hold fundraisers in order to make ends meet. I’ve playing with these thoughts since 2002 when … Continue reading “Happy new year: MetaBrainz reaches sustainability!”

Now that the holidays are over its time to get back to work and kick MusicBrainz work back into high gear!

Its been my goal for many moons to make MetaBrainz self sufficient in order to not have to hold fundraisers in order to make ends meet. I’ve playing with these thoughts since 2002 when it looked like a business model could emerge with which to build a non-profit. And finally near the end of 2007 we reached this exciting milestone. Taking on new customers like Last.fm and the BBC made all the difference — we now have slightly more money coming in than we’re spending. We’re far from rich, but we’ve never been on more solid footing than this.

This gives me lots of hope for 2008! My plans now include trying to raise even more money so we can hire a full time server developer in order to make more progress on fixing bugs, writing new features and finally breaking ground on NGS. There is lots to do to keep things moving, but reaching sustainability is a strong vote of confidence from the universe that MusicBrainz and MetaBrainz may be up to something that could really work.

And that makes me smile. Onward!

P.S. Year end finances are more work to wrap up than usual. So, it will take longer than normal for me to post the finances for December and all of 2007.

Search server fixed.

UPDATE: The main search functions are back! FreeDB search will be offline until sometime tomorrow. Our search server finally gave up the ghost and its dead. We’re bringing up a back-up server to take over for it, but it will take another 8 hours for the indexes to be complete. Sorry for the inconvenience! In … Continue reading “Search server fixed.”

UPDATE: The main search functions are back! FreeDB search will be offline until sometime tomorrow.

Our search server finally gave up the ghost and its dead. We’re bringing up a back-up server to take over for it, but it will take another 8 hours for the indexes to be complete.

Sorry for the inconvenience!

In the meantime, you can donate some money!

UPDATE: by Dave at 1730 UTC

Until about 1715 UTC the search server was only available to the master site – this is now fixed. Sorry for the additional inconvenience, and thanks for bearing with us.

Previously Dave said:

The server server seems to be a little sick again today – the load average was over 100 just now. As a result I’m trying to reboot the server now, and I’ve put request rate-limiting in place on the main server, so you may find that various things aren’t working at the moment. 🙁

Watch this space for updates.

Two new editing suggestion reports available

Sergey and Wendell at MusicIP have been working on two new reports: Tracks attached to the wrong artist: This list contains tracks with similar names, which have been connected to artists which may be ambiguous – often this indicates an error in either the track assocation, or the artists themselves. To resolve the conflict, tracks … Continue reading “Two new editing suggestion reports available”

Sergey and Wendell at MusicIP have been working on two new reports:

  1. Tracks attached to the wrong artist: This list contains tracks with similar names, which have been connected to artists which may be ambiguous – often this indicates an error in either the track assocation, or the artists themselves. To resolve the conflict, tracks assigned to the incorrect artists should be moved to correct artists, or the artists should be merged if they represent the same musical source.
  2. PUIDs attached to the wrong artist: This list contains tracks which have the same PUID, but have been connected to conflicting artists. To resolve the conflict, PUIDs assigned incorrectly should be removed, tracks assigned to the incorrect artists should be moved to correct artists, or the artists should be merged if they represent the same musical source.

Thanks to Sergey and Wendell for creating these reports!

Last.fm contract signed!

Just a quick update on the contract with Last.fm: Last.fm has signed the contract with MusicBrainz and will now be able to integrate MusicBrainz data into their service once again. Three cheers! Thank you much to Spencer at Last.fm for pushing this through!

Just a quick update on the contract with Last.fm: Last.fm has signed the contract with MusicBrainz and will now be able to integrate MusicBrainz data into their service once again.

Three cheers!

Thank you much to Spencer at Last.fm for pushing this through!

Looking for free hosting for a UK MusicBrainz Mirror

Powernet in Milton Keynes is currently graciously hosting our UK mirror, but unfortunately that free ride will soon come to an end. Thus, we’re looking to find a new home for a 1U server in some colo in the UK. The mirror does not take too much bandwidth at the moment — if the bandwidth … Continue reading “Looking for free hosting for a UK MusicBrainz Mirror”

Powernet in Milton Keynes is currently graciously hosting our UK mirror, but unfortunately that free ride will soon come to an end. Thus, we’re looking to find a new home for a 1U server in some colo in the UK. The mirror does not take too much bandwidth at the moment — if the bandwidth goes above a 500kbit/s I would be greatly surprised.

If you have some spare space in a colo in the UK and would like to help out by hosting a mirror server, please leave a comment!

Thanks!

UPDATE: The UK mirror will go away tomorrow, Nov 8 at about Noon, local time.

In case you didn't catch it: Picard != Picard QT

Given the number of clueless/negative comments on the TRM announcement, I feel it necessary to clue people in to the fact that Picard QT’s interface has been revamped. The workflow has been greatly improved, the user interface is more stable and it does an amazing amount of work for you before you need to use … Continue reading “In case you didn't catch it: Picard != Picard QT”

Given the number of clueless/negative comments on the TRM announcement, I feel it necessary to clue people in to the fact that Picard QT’s interface has been revamped. The workflow has been greatly improved, the user interface is more stable and it does an amazing amount of work for you before you need to use acoustic fingerprinting.

If you feel the need to trash months of hard work that Lukas has done, please at least download it and try it before you make an idiot of yourself.