Author: ruaok

Google Summer of Code starts again!

Apparently we’ve all made another trip around the sun, because Google’s Summer of Code starts off again today. We’ve applied to be a participating organization again and we’ve put together our ideas page. This page lists the ideas we have for what students can work on over the summer.

However, we find that students who bring their own ideas and inspiration tend to work out better in the program. So, if you are a MusicBrainz fan and you are a current university student who can program, you should consider participating. You’ll need to make sure qualify by Google’s rules.

If you do qualify, can program, and are interested but you’re unsure where to go next, please come visit us in the #musicbrainz channel on irc.freenode.net. We can help you take your ideas and shape them into something that would be workable for a summer project.

Concert listings on MusicBrainz?

During the last summit we spoke about listing concerts on MusicBrainz artist pages. So far we’ve had three concert listing providers approach us about listing concerts on our artist pages.

I generally hate the idea of advertising and I find the idea of advertising on MusicBrainz as very distasteful. But, being alerted to a concert in your area while browsing MusicBrainz seems like a very valuable addition to our site, especially if people are willing to provide the data for us for free!

So, I have two questions for the general MusicBrainz community:

  1. How do you feel about concert listings on MusicBrainz?
  2. Should we make the concert data providers pay for us showing their concert information? Since three companies have already approached us, it seems that we have leeway to ask for payment for this service. (This then would allow us to pay for some of the programmers who slave over making MB better for everyone).

Regardless of payment or not, we would rotate providers while showing listings to give every provider a fair number of impressions for their concert listings.

Please post your comments below!

MetaBrainz Foundation Annual Report for 2009

Introduction

In 2009 MusicBrainz placed a greater emphasis on working on our Next Generation Schema (NGS) and not focusing on business aspects of MusicBrainz as much. This turns out to have been a good strategy as 2009 was a lousy year to do business; the entire world economy was depressed. Many other non-profits had a really hard year, struggling to keep afloat. Fortunately the MetaBrainz Foundation managed to remain in the black for its fifth year in a row!

The most important aspect of work for MusicBrainz was NGS and after several years of planning we really kicked NGS development into high gear in May. Our employee Oliver Charles and our top contributor Lukáš Lalinský teamed up to tackle NGS as their combined Google Summer of Code project. Over the summer the codebase that Oliver had been tending for nearly a year already got another overhaul. And, by the end of summer a strong new MusicBrainz Server emerged! Paul Taylor took over the development of the Search Server and delivered a project that used 1/3 of the computing power than the code it replaced. In 2009 we only had one server release that brought us Release Groups and ISRC support — we had planned to have many more releases, but instead we opted to work on NGS full speed ahead.

2009 was also a year of many other changes: We switched from the MoinMoin wiki to MediaWiki; we started moving from Trac to Jira for bug tracking and we started the process of moving from Subversion to Git for the main server codebase. And Kuno Woudt quit his job and prepared to come to work for the MetaBrainz Foundation in February 2010. In November 2009 we also gathered in Nürnberg Germany for the 10th MusicBrainz summit. Finally, Carol Smith of Google became our Secretary/Treasurer in December!

In many ways 2009 was a transformative year for MusicBrainz with an amazing amount of effort spent on improving the site for future growth. Sadly we didn’t see many fruits of our labors in 2009, but 2010 promises to be amazing as we roll out NGS!

Profit & Loss

In 2009 the foundation took in $103,387.77 and spent $97,685.50 for a total excess income $5,702.27. This excess income is down from $31,724.40 from the prior year; there were two primary reasons for this decrease in revenue: 1) The economic climate of 2009 was terrible and 2) the BBC had a 6 month delay in renewing their data license contract, which accounts for $9000 lost revenue that we’ll collect in the first half of 2010. Given these two factors, I’m pleased with the outcome for 2009!

Income

Direct Donations $30,170.00
PayPal Donations $7,631.97
GSoC Mentor Income $1000.00
Live Data Feed licenses $42,078.50
CC Data licenses $2,800.00
Interest $517.53
Amazon Associates $1,112.21
CD Baby Affiliate $19.00
Tagger Affiliates $17,842.96
Reimbursements $215.60
Total Income:   $103,287.77

Expenses

Bank fees $366.67
PayPal fees $1,243.34
Rent $2,856.00
Hardware $3,152.56
Travel $2,229.15
Internet $100.00
Marketing $100.00
Development $67,231.41
Gifts $16.81
Events $1,481.18
Hosting $15,475.00
Utilities $13.42
Filing Fees $50.00
Entertainment $300.16
Insurance $2,025.00
Accounting $1,000.00
Shipping $44.71
Total Expenses:   $97,685.50

The Profit & Loss shows:

  • It costs $18,627.56 to host MusicBrainz in 2009 — this includes hardware and hosting costs.
  • PayPal donations, which are end-user donations, come to $7,631.97 about 7.3% of our overall income. This is down from 10.5% in 2008, presumably because of the horrible economic times. Counting Tagger Affiliate income (which is income from end-users via our Tagger Affiliates) our end-user contributions totaled $25,474.93, which exceeds our hosting costs by $6,847.37. This means that we’re still well within our goal of having end-user contributions cover our hosting costs — even in a down economy.
  • Development costs in the form of salaries cost $67,231.41 for both Robert Kaye and Oliver Charles. This figure will increase drastically in 2010 since Kuno Woudt joined the team. This presents the largest financial challenge facing the foundation in 2010.
  • $44,878.50 was earned in license fees in 2009. Counting the BBC license fees that should have been collected in 2009 we would be at $53,878.50 as compared to $62,872.56 in 2008. Our license fee income represents 52.1% of our income, up from 49.7% in 2008.

In 2009 the foundation spent $18,627.56 on hosting and hardware costs and served out 2.1 billion web hits and 1.3 billon web service hits. Calculating a cost per hit, we find that we spent $8.66 per one million web hits and $14.37 per one million web service hits. These values are down significantly from the 2008 values of $16.21 and $26.58 respectively.

Balance Sheet

The balance sheet for the end of 2009 showed the MetaBrainz Foundation with $71,309.67 retained earnings, a net income of $5,702.27 and a total of cash assets of $77,011.94.

Traffic

The following chart shows our overall web traffic to musicbrainz.org for 2007 – 2009:

Musicbrainz Traffic 2007-2009

(click on the image to see a larger version)

The blue line represents the overall number of hits to musicbrainz.org. The red line shows how many of the overall hits were web service (API) hits; In 2009 60% of our hits were web service hits, which stays roughly consistent with our figure of 60.9% in 2008. Please note that this does not mean that 40% of our hits were full pages that were served; the total hit count includes images and other static content required to load a web page. It appears that about 10%-15% of our total hits were complete pages we served.

Top contributors

Top editors

1. drsaunde 80921
2. murdos 45003
3. nikki 41438
4. brianfreud 40312
5. voiceinsideyou 39372
6. NAvAP 29648
7. ojnkpjg 28369
8. reosarevok 28132
9. crazee_canuck 25100
10. MeinDummy 24914
11. fred576 24891
12. gnu_andrew 22367
13. salo.rock 21668
14. mr_maxis 21620
15. HumHumXX 20082
16. Senax 19867
17. cooperaa 17980
18. jesus2099 16320
19. Dr. Default 15601
20. chabreyflint 13297
21. MrH 13135
22. zos18 12524
23. KRSCuan 12179
24. Fabe56 11544
25. mudcrow 10603

Top voters:

1. voiceinsideyou 47841
2. murdos 34898
3. chabreyflint 31919
4. drsaunde 30487
5. MClemo 28067
6. KRSCuan 22779
7. Fabe56 19651
8. ojnkpjg 18192
9. nikki 16656
10. bogdanb 15239
11. wheelpoint 13891
12. chiark 13299
13. SuicideScrub 12843
14. NAvAP 11362
15. PhantomOTO 11238
16. teleguise 10963
17. Plagueis 10924
18. salo.rock 10724
19. MeinDummy 10402
20. ulugabi 10112
21. DrMuller 9818
22. yoursvivek 9674
23. fred576 7924
24. cooperaa 7787
25. pankkake 7154

A big thank you to all of the editors/voters who contributed! MusicBrainz would be nothing without your hard work!

Server farm

MusicBrainz has grown to 14 machines in service:

From the top, going down:

  • moose: Our database server
  • scooby: Our aging catch all server: blog, forums, mailing lists, etc
  • catbus: Raw database server (raw tags, collections, etc)
  • bender: Former TRM server, now idle cold spare machine
  • blik: memcached
  • stimpy, dexter: web service servers
  • cartman: Search index builder
  • wiley: New catch all server: SVN, git, jira, trac, backups
  • tails: Web server
  • lenny/carl: Redundant network gateways
  • misty/jem: Search servers

MusicBrainz currently requires somewhere between 5 and 6 mbits of bandwidth per second and draws 19 Amps of current for a power consumption of about 2,090 Watts. MusicBrainz physically occupies 20Us of space (half of a rack) at Digital West in San Luis Obispo, CA. Additionally, we have a mirror server was hosted at Last.fm in London and a test server hosted by Amplifind Media Services in Los Angeles.

Words of Appreciation

2009 was often challenging; trying to retool a large web site on a shoe-string budget is never fun. But in the end 2009 was the best year in MusicBrainz history — never before did we accomplish as much as we did in 2009!

Oliver Charles, Kuno Woudt, Paul Taylor, Lukáš Lalinský, Aurélien Mino, Nikki, Pavan Chander, Phillip Wolfer and Brian Schweitzer — your contributions to MusicBrainz cannot be underestimated! And, Dave Evans, continues to be our behind the scenes hero — MusicBrainz keeps running because of you! Also a big thanks to our board of directors who keep me in line and ensure that we keep out of legal trouble. I’d also like to thank Google and the entire Open Source Programs Office for supporting MusicBrainz in 2009 again.

Onward to an exciting 2010!

LyricWiki and MusicBrainz integration!

LyricWiki and MusicBrainz are two community-powered sites about music which share some of the same passionate users. LyricWiki contains song lyrics and MusicBrainz contains metadata (detailed info) about music.

As of today, the two projects are going to be able to work together better than ever. Now that LyricWiki has obtained licensing, MusicBrainz can now link to song lyrics without worrying about legal issues. LyricWiki has given explicit permission to MusicBrainz to do so and MusicBrainz has added LyricWiki links (as an AR type) to their standard.

Linking from LyricWiki to MusicBrainzLyricWiki to MusicBrainz

To link to an artist, track, or release on MusicBrainz, put “musicbrainz= [UUID]” into the footer (ArtistFooter, AlbumFooter, or SongFooter) template. For [UUID], put in the MusicBrainzIdentifier. This has been available for quite a while, but hopefully there will be many more links now that we’re working more closely.

Linking from MusicBrainz to LyricWikiMusicBrainz to LyricWiki

To link a release or a track to LyricWiki, copy the LyricWiki album or track URL and then navigate to the proper release or track page in MusicBrainz. Be sure to be logged in and then click on “Relate to URL” link. Follow the instructions on the next page to add the LyricWiki URL to MusicBrainz.

This is an exciting opportunity for the two projects to be more closely linked together. We’re looking forward to it!

– Sean Colombo & Robert Kaye

Google ups its sponsorship of MusicBrainz in 2010!

I’m proud to announce that Google has decided to sponsor MusicBrainz once again in 2010! Google has pledged to donate $40,000 to the MetaBrainz Foundation this year! Tons of thanks to Chris DiBona, Leslie Hawthorn and Carol Smith (yes, the Carol Smith who is our treasurer/secretary!) of the Google Open Source Programs Office for your generous donation!

The extra $10k above the donation from last year aims to help us pay for the Kuno (Warp)’s paycheck as we work to finish NGS. However, Google isn’t likely to increase the yearly amount donated to MusicBrainz unless MusicBrainz can come up with some sort of way of giving some value back to Google.

I would like to finally allow Google to crawl the MusicBrainz web pages so that the rich information we have on our site finally gets exposed to Google’s search. But, that doesn’t really provide much value to Google — the value in doing this is ours.

In what ways can we do something that gives value back to Google for their continued support? Please post your ideas in the comments!

Testing the NGS live data feed

If anyone would like to test out the NGS replication, which I’ve setup from the test server, follow these instructions:

  • Download and install the mb_server source code from SVN trunk. Follow these instructions.
  • Set your server type in DBDefs.pm to RT_SLAVE
  • Download and import this dataset.
  • Insert this required row into the database, using our psql program: 
    cd /admin
./psql READWRITE
insert into replication_control values (1, 12, 36169, '2010-01-23 00:00:02.604674+00');
  • Now run admin/replication/LoadReplicationChanges a few minutes after the hour to keep up to date with the data on the test server. Please note that this system may not be stable yet and that we will occaisionally load new data on our test server, which will require you to reload the data on your server.

Good luck!

Database on test server updated

The database on the test server has been updated with the latest data and search indexes to match the latest work that ijabz’ has done. This update was in preparation for testing the replication system, which I’ll invite the general public to try in a few hours.

IMPORTANT NOTE: The passwords on the test server are now the same as the passwords from the main server as of January 23.

Announcing NGS Beta 1!

After many years of discussion, planning and hard work, we’ve finally reached the much anticipated Next Generation Schema Beta 1 release!

For all the information on this release, please take a look at our release notes. Then head over to the test server and play with it!

Most important of all, please note that we are using Jira to keep track of bug for the MusicBrainz server now. Please do not enter bugs for Beta 1 into the old (trac) bug tracker!

Many thanks to the countless people who have helped make NGS a reality!

NGS Beta 1 is coming: Monday January 18

I’m pleased to announce that we’re almost ready to release beta 1 of the much anticipated Next Generation Schema. All of the important core functions are complete and we’ve even done a first round of sanity checking to make sure what what we release will be useful. We won’t have all of the features for NGS done — there will be some things that we simply won’t have finished by then.

But, this release is important since we feel that we want the community to help us take a look at NGS and help us flush out problems before we polish NGS for release.

We’ll have loads of more details and even some documentation for you by next monday. Stay tuned!

MetaBrainz hires Kuno Woudt (warp) as a full time developer!

I’m pleased to announce that the MetaBrainz Foundation has hired its first full time developer! On Februrary 1, 2010, Kuno Woudt will join Oliver Charles on the MusicBrainz development team and both of them will focus to finish NGS as soon as possible. With Kuno we are effectively tripling the engineering capacity we have dedicated to writing code for NGS. This will free me up to focus more on managing the development process as well as focusing on more business development issues. An increased focus on business should allow me to find the extra income to pay Kuno and to keep us in the black!

I’m excited by this development — having dedicated developers working on finishing NGS is what has been needed for several years now. I’m very much looking forward to releasing NGS and then having a number of smaller releases in 2010!