TRM Database Pruned

It’s TRM pruning time again.

At about 2300hrs UTC on January 25th the TRM database
was “pruned” again. 
As we did last time, we removed all TRMs apart from the ones attached to
MusicBrainz tracks, this time using the additional criterion that
the TRM had to have been looked up at least twice. 

Just before the prune we had about 3633572 TRMs; after the prune we had about 1898435. 
For pretty pictures illustrating this, check out our
MRTG pages.

See also prune 1,
prune 2
and prune 3.

Server Made Simple

In the past, installing a MusicBrainz server has always been a testing and lengthy task, without any clear-cut set of instructions to follow. Over the last few days I’ve been putting together a method of installing mb_server, starting from scratch, using Debian. Installing the server just got a lot easier.

In the past, installing a MusicBrainz server has always been a testing and lengthy task, without any clear-cut set of instructions to follow. Over the last few days I’ve been putting together a method of installing mb_server, starting from scratch, using Debian. Installing the server just got a lot easier.

Continue reading “Server Made Simple”

AR todo list

If you’ve been lusting for the new Advanced Relationships feature in MusicBrainz, take a look at our AR Todo list. We need to complete this list of tasks before we release AR on the world. We’re now officially done with the features, but we still need to do some performance tweaking and behind the scenes … Continue reading “AR todo list”

If you’ve been lusting for the new Advanced Relationships feature in MusicBrainz, take a look at our AR Todo list.

We need to complete this list of tasks before we release AR on the world. We’re now officially done with the features, but we still need to do some performance tweaking and behind the scenes futzing.

If you’d like to play, please go to the test server!

But keep in mind that the relationships on the test server will change! (They should be more complete and suck less for the first release.)

TRM Database Pruned

The TRM database has been pruned again, making the system much faster and more reliable again.

At about 2200hrs UTC on December 22nd the TRM database
was “pruned” again.
As we did last time, we removed all TRMs apart from the ones attached to
MusicBrainz tracks, where the TRM had been looked up at least once. 

Just before the prune we had about 3632598 TRMs; after the prune we had about 1989632. 
For pretty pictures illustrating this, check out our
MRTG pages.

See also Prune 1 and
Prune 2.

Advanced Relationships is here!

If you’ve been watching MusicBrainz from the sidelines waiting for the fabled Advanced Relationships (AR) feature, wait no more! We have begun the first phase of testing for AR, on our staging server. Before you ask questions or start giving us feedback, please read the AR announcement for more details on this new feature.

If you’ve been watching MusicBrainz from the sidelines waiting for the fabled Advanced Relationships (AR) feature, wait no more! We have begun the first phase of testing for AR, on our staging server.

Before you ask questions or start giving us feedback, please read the AR announcement for more details on this new feature.

Server Troubles

Recently the server has been hit by patches of instability – large load spikes, running out of memory, and processes getting killed here and there. When the most recent out-of-memory condition occurred (last night) the SSH server was one of the processes which got killed, which is why the server had to be rebooted a … Continue reading “Server Troubles”

Recently the server has been hit by patches of instability – large load spikes, running out of memory, and processes getting killed here and there. When the most recent out-of-memory condition occurred (last night) the SSH server was one of the processes which got killed, which is why the server had to be rebooted a little while ago.

I’m fairly sure I more or less know what’s been causing the problems, and have made a few changes to try to reduce the chance of it happening again.

One of the worst causes of the problem is looking up a TRM with a large number of tracks. The worst TRM by far for this is the “silence” TRM, with (currently) over 900 tracks. As a result I’ve had to, for now at least, disallow lookups on this TRM – doing so will now simply return an error. Sorry 😦 Maybe it can be made to do something more helpful in future.

The other change is that if you do a lookup on any TRM which has more than 100 tracks then only 100 of those tracks will be returned. However so far there are no TRMs (except “silence”) with over 100 tracks, so this won’t affect anyone, yet. As the data grows, it will though.

Sorry for any inconvenience caused (hey, I’m apologising again. This is getting to be a habit). But I’m sure you’d rather have a server which doesn’t keep crashing and locking us all out. Hey ho.

Server Updates

“Add Disc ID” moderations, and Annotations.

Changes mainly of interest to MusicBrainz Users

“Add Disc ID” Moderations

Whenever a disc ID is added to an existing album, it is now tracked
via an “Add Disc ID” moderation.  This applies both to disc IDs added
via the “CD lookup” interface (in which case the moderation is credited to
whoever performed the lookup), and also to those added as a result of a
FreeDB lookup (which fall under the “FreeDB” moderator). 
“Add Disc ID” moderations are not used in the case where
an album and a disc ID are added at the same time.

Annotations

Annotations allow you to add notes to artists and albums. 
See How Annotations Work
and the Annotations FAQ
Thanks to Matthias Friedrich for building the foundations of this feature.

Bugs and RFEs Closed

Dave Evans

Try out the latest server features

I’ve opened my development box up to the world – you can see what I’m working on at http://dev-mb.djce.org.uk/.  The mb_server I have running at the moment, for example, implements “Add Disc ID” moderations.  Play nicely!

I’ve opened my development box up to the world – you can see what I’m working on at http://dev-mb.djce.org.uk/.  The mb_server I have running at the moment, for example, implements “Add Disc ID” moderations.  Play nicely!

Lucene web service

In the last two weeks I managed to combine working on MusicBrainz, creating a new open source project and earning money to pay the bills! This is quite rare these days, so I am pleased all around. As some of you may know, I have been doing contract work for CD Baby. When Derek, the … Continue reading “Lucene web service”

In the last two weeks I managed to combine working on MusicBrainz, creating a new open source project and earning money to pay the bills! This is quite rare these days, so I am pleased all around.

As some of you may know, I have been doing contract work for CD Baby. When Derek, the owner and lead geek at CD Baby, asked me what MusicBrainz does for searching, I launched into a long cheerleading rant about Lucene. I managed to convince Derek that Lucene is the way to go, and to convince him to sponsor the open source development of the new Lucene Web Service. Luckily Derek agreed that as long as the project was going to be available under the BSD license that he would agree to open source the work.

Triple cheers for Derek and CD Baby please!

So, the web service is now done and I’ve applied for a new project on SourceForge — once that is approved, I will release the source code for everyone to check out. I’ll post another message here when that is complete.

If you’d like to check out the working web service, try this link.

TRM Database Pruned

The TRM database has been pruned again, making the system much faster and more reliable again.

At about 1930hrs UTC on November 4th the TRM database
was “pruned” again (see the previous time
for more information about this). 
This time we removed all TRMs apart from the ones attached to
MusicBrainz tracks, where the TRM had been looked up at least once. 
This is a slightly more aggressive prune compared to the previous time. 

The TRM database is now about
40% of the size it was before, which (like last time) means that
it now fits easily into the server’s memory, so the server as a whole
runs quickly
and reliably.