Category: MetaBrainz

GSoC 2019: JSON Web API for BookBrainz

The time has come to wrap up the very productive and learning summer of the last 3 months as a GSoC student with MetaBrainz.

Hello Everyone!!

I am Akhilesh Kumar, a recent graduate from the National Institute of Technology, Hamirpur, India. I have been working on BookBrainz for MetaBrainz Foundation Inc. as a participant in the Google Summer of Code ’19. It has been an amazing experience and I’ve learned a lot over the summer. I was mentored by Nicolas Pelletier (Mr_Monkey on IRC) during this period. This post summarizes my contributions to the project and the experiences that I had throughout the summer.

Continue reading “GSoC 2019: JSON Web API for BookBrainz”

We were sued by a copyright troll and we prevailed!

must be monetary compensation

On August 9th, 2018 we were served with a United States federal copyright infringement lawsuit over a handful of images displayed on our musicbrainz.org artist pages. These images were made available by Larry Philpot, a photographer, on Wikimedia Commons and we “deep linked” to the images (that note the license details and attribute the images to their creator) from our artist pages, in accord with the license terms.

The MetaBrainz Foundation prides itself in treading carefully in legal matters and so we were surprised to receive a lawsuit of this nature. All allegations in the suit were deemed false by our legal team. If you wish to find out more about this lawsuit, we encourage you to read the documents that were served to us.

Upon being served with the lawsuit, MetaBrainz contacted our legal guardian angel: Ed Cavazos of Pillsbury Winthrop Shaw Pittman LLP, who has been watching over the foundation since its inception. Ed proposed our case to the pro-bono committee at Pillsbury and to our great pleasure the case was accepted! Pillsbury officially became our legal representatives in defending us in this lawsuit.

Ed assembled a team (Brian Nash, Ben Bernell, Sarah Goetz) who fired off an immediate response to the lawsuit. The team filed a timely response with the court and then began a lengthy journey of educating themselves on how MetaBrainz conducts business, how it hosts its websites, and how these websites came into existence. Over the course of many emails and calls, MetaBrainz produced volumes of conversations, bug reports, Git commits and various other forms of substantiating information that the legal team used to form a strategy.

Our legal team operated on the basis that “the best defense is a good offense”. The team’s filing showed that the accusations were unfounded and went on to question the motives and methods of the plaintiff, who has a history of taking legal action against Creative Commons users. In these legal actions he claims that the users have violated Creative Commons licenses, according to narrow, non-customary interpretations of the obligations and limitations set out in CC licenses. It didn’t take long for the plaintiff to feel our pressure and decide to cut their losses. On February 28, 2019 the lawsuit was dismissed with prejudice!

Now that this is behind us, the MetaBrainz Foundation had to figure out what to do about showing Wikimedia Commons images on our websites. We talked with both the Wikimedia Foundation and Creative Commons to discuss what had happened. We learned that both Wikimedia and Creative Commons had started their own processes to examine and address the issues that led to the lawsuit being filed against the MetaBrainz Foundation.

We’re looking forward to seeing firm and decisive action from our friends at Creative Commons and Wikimedia, before other people and nonprofits are put in harm’s way by what in our opinion constitutes unacceptable, predatory misuse of CC licenses and Wikimedia Commons. MetaBrainz has made sure that CC and Wikimedia know about our experience and now we’re returning our focus to our core mission.

While we wait for Wikimedia Commons and Creative Commons to take action on this, we will not reinstate artist images or include any images that link to Wikimedia Commons. We prevailed in this lawsuit and thanks to our pro-bono legal team we suffered no harm. Being dragged through lengthy court proceedings by trolls hoping to make an example of us could exhaust our reserves and leave us broke — but that won’t stop us from vigorously defending ourselves. We are not going to let a bully push us around.

That’s about all we can say about this. The court filings speak volumes about the merits of the case and the problems of predatory abuse of CC licenses. It sucks to be the target of a pointless, predatory lawsuit. We’ve always been very careful about staying on the right side of the law, and we’re prepared to go to court to prove it, even if we can’t get pro-bono counsel.

The MetaBrainz Foundation owes a debt of gratitude to Ed Cavazos, Brian Nash, Ben Bernell, Sarah Goetz and Pillsbury Winthrop Shaw Pittman LLP. We cannot overstate how fortunate we are that the team came to our rescue at a very critical juncture. Thank you to the whole team and Pillsbury Winthrop Shaw Pittman LLP. Thank you!

We would also like to thank Cory Doctorow (one of our directors) for initiating and participating in many conversations. Not only was Cory’s advice critical in dealing with the lawsuit, but it was Cory and the EFF who connected us to Ed Cavazos in the first place, 15 years ago. Thank you!

I personally would like to thank Nicolás Tamargo and Michael Wiencek for their support in digging for documentation to support our side of the case. Thank you for your tireless efforts!

Finally I would like to thank our board of directors for their support in this process. Thank you Cory, Matthew, Rassami, Paul and Nick!

UPDATE: A few people have requested for us to publish our response to the lawsuit. On 28 September 2018, we filed this response with the court.  That is the only public filing we made — the lawsuit was dropped on February 28, 2019 as a direct result of private conversations with the plaintiff.

Google donates $10,000 in cloud computing credits. Thank you!

The Google Open Source Programs Office continues to support MetaBrainz in a number of ways and most recently they donated $10,000 in credit toward their cloud services. Thank you Google!

This credit allows us to run some services in the cloud to round out primary hosting setup — this gives us a some redundancy and allows us to not keep all of our critical eggs in one basket. We can also give our open source developers Virtual Machines from time to time, since a lot of our projects are very data heavy. Having access to a fat VM can sometimes turn a really frustrating project that makes your laptop melt into a project that is satisfying to watch chug along.

Thank you again, Google, the Open Source Programs Office and in particular, Cat Allman!

MusicBrainz Schema change upgrade downtime: 17:00 UTC (10am PST, 1pm EST, 19:00CEST)

Hi!

At 17:00 UTC (10am PST, 1pm EST, 19:00CEST) we will start the process of our schema change release. The exact time that we plan to start the change will depend on how long it takes to finish our preparations, but we expect it to be shortly after 17:00UTC.

Once we start the process we will put a banner notification on musicbrainz.org and we will also post updates to the @MusicBrainz twitter account, so follow us there for more details.

After the release is complete, we will post instructions here on how to upgrade your replicated MusicBrainz instances.

Google Summer of Code 2019: Accepted students and their projects

The accepted students for Google Summer of Code have just been announced! We’re please to announce that Akhilesh Kumar (BookBrainz), Aidan Lawford-Wickham (AcousticBrainz), Vansika Pareek (ListenBrainz), Anirudh Jain (MusicBrainz), amCap1712 (MusicBrainz) and Shamroy Pellew (CritiqueBrainz) have been accepted on behalf of the MetaBrainz Foundation!

To find out more about the accepted students and what they will be working on, please take a look at the list of accepted projects.

This year was quite challenging to decide which students to accept. We had more good proposals than we could accept — which is quite heartbreaking, since we hate having to turn away good proposals. Still, we have a very good spread of students across our projects and we’re quite excited for Summer of Code this year.

Thanks to everyone who applied, all of our mentors and of course, Google’s Open Source Programs Office for making Summer of Code a reality.

MetaBrainz team changes, autumn 2018

Hello!

The only constant in the world is change, right?

First off, the somewhat sad news: Sambhav, AKA samj1912, has left MetaBrainz the team as a contractor and has moved to London. The upside of this news is that he will continue to work on Picard for us and will remain a part of our team as a volunteer, but his presence will not be quite as intense as before. Thank you for your hard work these past months, especially for finishing the impossible Solr search project!

With Sambhav’s departure and our improved finances, I’m proud to announce that we’re taking on two new contractors!

Nicolas Pelletier AKA Monkey: You may remember the talented Monkey from when we designed our new logos. He was the designer who created the logos and our new bootstrap theme that adorns most of our pages now. Working with Monkey was straightforward, effective and the results were great, so when he expressed interest in working on BookBrainz, I was pleased to hear this news. Monkey will be working for us full time and spending 75% of his time on BookBrainz and 25% of his time to help with design and UX work for the rest of our projects. In the next blog post I’ll talk more about BookBrainz and what we can expect from that project in the future.

Nicolás Tamargo AKA Reosarevok: Reosarevok is no stranger to our community — he’s made 1.7M edits to MusicBrainz, is our Style BDFL and answers all of our support@ emails. He’s been learning more programming and asked to be part of the MusicBrainz team part time. We agreed to give this a go and in the short term he will be focusing on genre support and helping with the React migration among other tasks. If this trial run works out, we’ll see about expanding his scope on our team.

Welcome on board Monkey and good luck with the new position, Reo!

 

 

GDPR compliance

The General Data Protection Regulation is a complex EU regulation that stipulates many points for protecting private data of users on the Internet. Even though this is an EU regulation, it has a worldwide impact due to the nature of the Internet. This regulation comes into effect today, May 25, 2018 and is the reason why so many companies have sent you mail in the past few weeks about updating their privacy policies.

The MetaBrainz Foundation with its collection of projects is also affected by this regulation. We’ve been learning and adapting our sites to be compliant with the regulation – sadly this regulation isn’t entirely black and white and there is an incredible amount of room left for interpretation of these rules.

The good news is that this regulation is roughly in line with our established practices: We’ve always held private information in a high regard and applied the sort of rules to ourselves as we wish to have our own private data treated. Luckily, this makes our compliance effort considerably easier. We’ve made two significant changes to how we treat your data and also adopted terminology as used in the GDPR in order to use the same languages that many other sites are now adopting. Please keep reading to find out the exact details of what we are doing to comply.

However, we do ask for your compassion and help in our process of complying with the GDPR. As we already mentioned, the GDPR is a complex set of rules that are not fully clarified yet. We’ve taken action on the steps that are clear to us and we’re following ongoing conversations on points that are in gray zones or unclear to us. We’ve made our best initial effort on compliance and promise to keep working on it as the picture becomes more clear. If you believe that we could improve our compliance, please contact us and let us know what we can do to improve. It would also help us if you could provide concrete discussion or examples to help us understand and take action on your suggestion.

Finally, below is the link to our GDPR compliance statement, implementing the regulations as we understand them and how they affect your data in our ecosystem. Where possible, we provide links for deeper understanding, links for you to examine our relevant code and links to tickets to follow the process of improving our compliance.

MetaBrainz’ GDPR Compliance Statement

 

MetaBrainz Foundation Board Meeting Minutes: May 18th, 2017 (and November 21st, 2016)

Here’s the second round of board meeting minutes. We have had board meetings both in November and in May, but the November board meeting only consisted of Robert, Sophie, Rassami, and Matthew meeting up in London approving the minutes of the prior meeting and agreeing to only hold 3 meetings in 2016. So we decided to kind of skip over reporting on that meeting and just include the summary when posting the minutes for the May meeting. Which I now just did. 😉

These minutes have been reviewed by the board but have not been formally approved at a board meeting yet, meaning that it is subject to change if the board feels something is wrong or missing. Also note that discussion related to personnel or confidential topics have been/will be redacted. Furthermore we may also add more text to the notes in order to give context and background, while not altering the meaning of these notes.

Let’s dive in!

MetaBrainz Foundation Board Meeting Minutes 2017-05­-18

Continue reading “MetaBrainz Foundation Board Meeting Minutes: May 18th, 2017 (and November 21st, 2016)”

Expanding our team

As the world comes back to life after the summer break, we’re making some changes and expanding our team. First, Roman Tsukanov has decided to not renew his contract with us. During his tenure with MetaBrainz, Roman adopted and released CritiqueBrainz and also wrote our new MetaBrainz web page, which is helping us bring in new supporters. His contributions have been far from trivial — thank you for your efforts, Roman!

Due in part to the new MetaBrainz web site, we’ve got more financial support than ever, and this allows us to replace Roman with two engineers! I’m please to announce that we’re hiring two of our Summer of Code students who just completed the program:

Sambhav Kothari AKA samj1912: Sambhav started hacking on Picard earlier this year and knocked Picard out of dormancy, working towards a new release and then making Picard his Summer of Code project. He completed his project with flying colors and is working towards a major upgrade of Picard. On the MetaBrainz team he is going to look after the new search infrastructure and the maintenance and bug fixing of our Web Service in addition to hacking on Picard. A full plate, for sure!

Param Singh AKA iliekcomputers: About the same time that samj1912 arrived, Param arrived. He expressed interest in working on ListenBrainz — he too dove right in and started making improvements. ListenBrainz had quite a ways to go before he could aim to make a Summer of Code project out of it. Param and I embarked on a journey to revamp and improve the stability of ListenBrainz, which culminated in us releasing the new ListenBrainz beta a few weeks ago. Since then he’s been focusing on his Summer of Code project, which is also now complete. On the MetaBrainz team Param will be looking after ListenBrainz and also the new MetaBrainz web site.

Both Param and Sambhav will officially start working on the MetaBrainz team starting October 1, but I strongly suspect we’ll see them around and hacking on the projects as has become the norm this year.

Welcome aboard Sambhav and Param!

 

ListenBrainz enters Beta stage

I’m pleased to announce that we released our first official beta version of ListenBrainz yesterday! As you may know, ListenBrainz is our project to collect, preserve and make available, user listening data similar to what Last.fm has been doing, but with open data.

In 2015 a small group of hackers gathered in London to hack on the first version of ListenBrainz alpha. We threw together a pile of new technologies and released the first version of ListenBrainz at the end of the weekend. In the end, we didn’t really like the new technologies (Cassandra, Kakfa) as both ended giving us a lot of problems that never seemed to end.

In 2016 we embarked on a journey to pick new technologies that we liked better and ended up setting on InfluxDB and RabbitMQ as backbones to our data ingestion pipeline. These tools were a good match for us, since we were already using them in production! Sadly, MetaBrainz’ move to our new hosting provider ended up sucking up any available time we had to devote to the projects, so progress was made in fits and starts.

Earlier this year Param Singh expressed interest to help with the project in hopes of joining us for a Google Summer of Code project. He started submitting a never ending stream of pull requests; slowly the project started moving forwards. Together we brought the codebase up to our current standards and integrated it into the workflow that we use for all of the MetaBrainz projects.

We proceeded to prepare the next version to be released at MetaBrainz’s new hosting facility and started a never ending series of tests. We kept pounding on the data ingestion pipeline, trying to find all of the relevant bugs and ways in which the data flow could get snagged. Finally the number of reported bugs relating to data ingestion dropped to zero and we managed to import 10M listens (a listen is a record of one song being played)!

That was our cue for promoting our pre-beta test to a full beta and unleashing it onto our production servers at our new hosting facility. Today we cleaned up the last bits of the release and we are ready for business!

What does this new release bring for you, the end users? Sadly, only a few new things, since most of the work has gone into building a stable and scalable system. We do have a few new things in this release:

  • Incremental imports from Last.fm — now you don’t have to do a full import any time you wish to import your latest listens from Last.fm. The importer knows when you last did and import and will work accordingly.
  • Last.fm compatible submission interface — with some system configuration changes you can submit your listens directly to ListenBrainz from any application with Last.fm support. (more info here)
  • Last.fm file import — if you have an old skool Last.fm zip file with your listening history backed up, you can now import it.
  • User data export — you can now download your own listens straight from the site, no waiting required.
  • Adaptive rate limiting on the API — our server now uses a modern rate limiting system. For details, see our API docs.

The good news is that Param is now working on his Summer of Code project that will add a lot of graphs and other critical elements for making use of this new data set. We hope to release new features on an ongoing basis from here on out.

Most importantly, we want to publicly state that ListenBrainz is now ready for business! We don’t plan to reset the database from here on out — this is the real deal and we plan to safeguard and make this database available as soon as we can. If you have hesitated with sending your listen histories to ListenBrainz in the past,  you should now feel free to send your listen information to us! If you are an author of a music player, we ask that you consider adding support for ListenBrainz in your player!

In a follow-up blog post I am going to write about how to start using ListenBrainz now — at the very least use it to back-up your Last.fm listening history!

If you find bugs with our latest release, please report them to our issue tracker. If you’re interested in this project and have questions for us, why not come and pop into our IRC channel or ask a question on our community forum?

P.S. The alpha version of ListenBrainz is still around.

P.P.S. We’ll have another cool announcement very shortly! Stay tuned!