The traffic to MusicBrainz is growing steadily! (steady growth is much better than exponential growth!)
We’ve recently added two more search servers, which are now humming along nicely. But now we’re seeing that our main web server is starting to get saturated, so we need to add more capacity. In order to use our existing machines effectively and to greatly increase the redundancy of our site, we’ve asked out board of directors to approve the purchase of two more servers. I expect this to be approved shortly.
In the meantime we’re getting more serious about people who are violating our very liberal terms of service that state that no single client instance is to make more than one Web Service call to MusicBrainz per second. We’ve been blocking mis-behaving clients more aggressively and have decided to block all clients that use this User-Agent id string:
Mozilla/4.0 (compatible; Win32; WinHttp.WinHttpRequest.5
If you’re using a MusicBrainz enabled application and all of the sudden it stopped working today, please leave us a comment below. We’d love to identify the offending application so we can contact the author of the application.
Also, the following IP address have been blocked from using MusicBrainz, either because of their client string or sending too many requests to MusicBrainz:
85.144.88.11, 69.181.45.228, 80.58.205.43, 205.209.78.232, 195.235.104.253, 85.240.173.204, 92.195.16.20, 89.241.123.24, 71.12.186.1, 204.17.31.126, 98.64.41.178, 77.188.36.140, 141.155.126.17, 217.225.157.236, 91.9.246.252, 66.91.250.27, 217.41.4.13, 83.76.160.150, 91.67.8.254, 88.191.58.11, 141.30.218.96, 88.88.124.26, 82.42.72.21, 91.9.230.72,
213.22.39.14, 86.0.180.87
In the last 16 hours, the following IPs have been over the limit:
| Total requests | IP address |
|---|---|
| 3187 | 58.170.57.43 |
| 3344 | 71.76.7.125 |
| 3523 | 92.193.11.17 |
| 3789 | 61.216.59.7 |
| 4495 | 67.180.35.184 |
| 4670 | 218.66.36.76 |
| 6521 | 91.47.225.62 |
| 7993 | 67.109.84.5 |
| 8558 | 91.47.135.154 |
| 189763 | 88.191.58.11 |
If you find your IP address on this list, please make a donation to the MetaBrainz Foundation and we’ll consider unblocking you after you assure us that you will not be making more than 1 request per second.
MetaBrainz Blog 
Good move! I for one am glad to see those abusing the service and taking resources away from other users finding themselves blocked.
My IP address is one of the ones you blocked (not one of the ones in the ‘Total requests IP address’ list) and I am using an application called Magic MP3 tagger. I guess this must be making web calls at a rate greater than 1 per second or is using the header you described, or perhaps even both. How can I find out?
I apologise for breaking the rules but I didn’t know! Honest! I don’t know how to make the application request at a slower rate either, so I guess I am back to manual tagging now. Annoying really since I paid hard earned money for the software… :o(
Would it help for me to contact the software provider too to complain that their software (or at least my copy of it ) has been blocked from using your excellent site?
Apologies again. Keep up the good work on a great service.
GP:
Which software was this? If you paid for it, then the software author must fix it! Please let us know which software this was!
(And if you didn’t write/modify the software in question, you didn’t do anything wrong!)
when my magic tagger tries to use the MB database it says could not establish internet connection however the freedb does work.
i am only using this program on one computer and i am sure i am using it fairly.
rob/mayhem – if the app is Magic MP3 Tagger (as the users note) that’s a concern, given it’s part of our tagger affiliate program and linked off the site!
Voice,
I think its not that drastic. The author of Magic MP3 tagger and I spoke about this months ago and he immediately released a new well behaved version. I think its mostly users using old versions that are causing problems. I’ve not gotten this confirmed yet, but if I do, I will post urging users to upgrade.
And I feel better that its a tagger affiliate that is causing problems. MP3 Tagger and Jaikos have been contributing enough money to cover our entire hosting costs. Can’t really complain about that can we? 🙂
Hello all,
I’m the author of Magic MP3 Tagger; I just checked the tagging application and I think the bandwith usage of MusicBrainz could actually partially result from the queries of Magic Tagger – if that’s the case, I deeply apologize for that, this has of course not been my intention.
I suppose that the reason of the problem (as far as Magic Tagger is concerned) is that the tagger enables users to cross-check unmatched albums online with MusicBrainz. In later versions of my software the user can do this in a batch process for the unmatched albums – which can result in many MusicBrainz requests if the user has a quite large music collection.
I can disable this automated uplook function again if it’s causing problems for MusicBrainz – I just thought it would be a nice feature that users can batch-correct their albums via MusicBrainz, that’s the reason I implemented it.
I’m going to tell all users that they should upgrade to the fixed version then. Since Magic Tagger already has quite many users, I think this will help (at least something) to reduce MusicBrainz’ bandwith usage.
I’m by the way still working on the new successor software of Magic Tagger for which I’m going to set up my own database server. As soon as this is done, there will be no more traffic from Magic Tagger clients for MusicBrainz any more. The release date isn’t sure yet, I hope it to be done in 2008.
I’d like to apologize again for those circumstances as far as they actually should result from Magic MP3 Tagger – I’m going to disable the automated lookup function immedeately and hope that this is going to help.
At last, I’d also like to remember that it should be considered that the total amount Magic MP3 Tagger donated to MusicBrainz is more than 10.500 EUR (more than 15.500 USD) within the last two years.
I’m looking forward for any responses.
Thank you,
Mathias (developer)
Mayhem, I have checked and I am using the latest version of Magic MP3 Tagger. I have been advise by the Author that the new (better behaved version) will be released very, very soon and will upgrade to that as soon as it is available.
Will this entitle me to gain access back to the Music Brainz database?
Thanks
I’ve fixed it now. The new release 2.2.4f of my tagger doesn’t allow automated MusicBrainz queries in batch mode any more. Mayhem, please let me know if this helps to reduce MusicBrainz traffic.
Mathias
Mathias,
Does 2.2.4f have a differente User-Agent string? And does it at all times respect the 1 query per second max rule?
Thanks!
GP:
Please download the 2.2.4f version and see if that works for you. Sorry for the inconvenience!
Mayhem,
the new version uses “Magic MP3 Tagger 2.2.4f” as User-Agent string now and respects the 1 query per second rule at all times. I’ve now enabled batch-processing for unmatched albums again (under respect of the 1 second rule, of course), but if it turnes out that this should also cause problems for MusicBrainz please notify me so that we can discuss what to do in order to assure the smooth operation of the great MusicBrainz service.
Emkey08:
Thanks very much!!
mayhem: ya, it was precisely because they’re such a good affiliate to the foundation that I thought it was concerning we might be blocking quite a few of their paying users. 🙂
Looks like some good collaborative work to trace and resolve the problem though, so three cheers all round :-]