We’re actually really going to take the HTTPS plunge!

Closing in on three years after stating that “We’re going to take the HTTPS plunge!”, we’re actually really going to do it now. 🙂

Most of our sites have forced HTTPS for some time (metabrainz.org, critiquebrainz.org, bookbrainz.org, listenbrainz.org), but there are still a couple of stragglers, notably musicbrainz.org and acousticbrainz.org.

For MusicBrainz, our beta site is now all HTTPS, web service and all. The main, non-beta musicbrainz.org will be going HTTPS-only except for what’s under /ws/ (ie., the web service) to allow taggers and other programs not currently using HTTPS some transition time. We do not currently have an ETA for when we will make the final jump to HTTPS-only on the MusicBrainz web service, as that partly depends on feedback from our web service users, which leads me to:

If you’re currently using the MusicBrainz web service, please try and switch your program to using beta.musicbrainz.org and see whether your program breaks or not and let us know the status of it. We are aware that some Python versions and MusicBrainz libraries do not support our setup, so while your program may fail now, it might simply be because of dependencies of your program not being updated yet and you might not need to do anything specifically on your end – however, some programs/libraries might need some updates, so the more people test and report back, the better we’ll be able to judge when we can go all-HTTPS-only on musicbrainz.org.

For AcousticBrainz, we now have a shiny new Let’s Encrypt certificate on https://acousticbrainz.org thanks to our systems administrator Zas! As a result, we are going to start redirecting all HTTP traffic to HTTPS on the AcousticBrainz website, including API queries.

In order to give everyone time to verify that their scripts correctly recognise and validate our Let’s Encrypt certificate, we are going to delay the redirect until July 1, 2016. On this date, any HTTP query will automatically be redirected to HTTPS. We will also enable HSTS, so that compliant browsers will redirect to HTTPS on the client-side.

If you have any questions about either the MusicBrainz or the AcousticBrainz transition, please ask.

Temporary autoeditor election procedure

Hello people,

In former days, new autoeditor elections were announced on the autoeditor mailing list that all autoeditors were automatically subscribed to. However, all our mailing lists, including the autoeditor one, died in September 2015 when the server they were hosted on took its last breath. This effectively halted the election of new autoeditors. It was always the plan that our new forums should be able to handle this, but our recent issues have meant development on the features necessary to completely replace the autoeditor mailing list has been slow. Thus Nicolás (reosarevok) and I had a talk about how to handle elections going forward, and we came up with this procedure:

  1. The proposer nominates the editor normally (starting an election), and then adds a post in the Autoeditors category of the forum linking to the election and asking for seconders. Like with the mailing list, this should also contain the proposer’s reasoning for proposing the candidate.
  2. Nicolás (reosarevok) will then mail out to all autoeditors with a link to the election topic (and possibly a link to this blog post).
  3. The proposer, Nicolás or me (or another autoeditor, if they’re faster) will update the thread once a 2nd seconder is found and the voting has started, and again when the voting has ended and the results are in.

We have added most autoeditors who are already signed up on the new forums to the @MB_Autoeditors group, but not all autoeditors have signed in to Discourse yet, and some have spaces or other “weird” characters in their username that make Discourse not able to parse them. If you find that you’re not in the @MB_Autoeditors group and you think you should be, please write a message to me (Freso) or Nicolás (reosarevok) via MusicBrainz and ask us to add you to the group. (Sending your message via MusicBrainz will let us know that it is indeed you/your account, so please don’t poke us on IRC or elsewhere about it.)

This is all obviously intended to be temporary, just until we’re able to get the process fully automated again. If you have any Ruby experience/know-how and would like to help out, please check out OTHER-248 (and possibly OTHER-254). There’s also MBS-8836 on the MusicBrainz server side for the Perl-istas.

Let us know if you have any concerns or questions about this (reminder: temporary) approach, either in the comments or on the forums. I personally hope it’ll work well enough to carry us through for a while longer until everything is ready.

GSoC 2016 students and projects

Google announced the final list of Google Summer of Code 2016 students and their projects yesterday. The list of MetaBrainz’ projects can be seen at our page on the GSoC site, but just for good measure, here’s the rundown:

MusicBrainz
Jeff Weeks (weeksio) returns to finish up the SOLR search server. We’re really hoping that this will be the end of our current search server woes. He will be mentored by the German duo of Ulrich Klauer (chirlu) and Rob Kaye (ruaok).
MusicBrainz Picard
Rahul Raturi (rahulr) will be working on improving searching MusicBrainz from within Picard, mentored by MusicBrainz’ senior developer Michael Wiencek (bitmap).
BookBrainz
Max Prettyjohns (QuoraUK) is going to try and take on adding gamification to our fledgling book/literature database. He will be supervised by the BookBrainz project leads and lead developers Ben Ockmore (LordSputnik) and Sean Burke (Leftmost).
ListenBrainz
Pinkesh Badjatiya (armalcolite) has pledged to tackle adding a much requested feature for our youngest project: implementing a Last.FM compatible submission API. Robert Kaye (ruaok) will be the one guiding him along.
AcousticBrainz
Daniele Scarano (hellska) will be spending the summer writing a toolkit for creating datasets, which should help researchers using AcousticBrainz. He will be mentored by MetaBrainz software engineer Roman Tsukanov (Gentlecat).
Kartik Gupta (kartikgupta0909) has set out to create an offline client for computing AcousticBrainz dataset evaluations. Alastair Porter (alastairp), the AcousticBrainz project lead, will be their mentor.
Goran Cetusic (cetko), our final student of this year, will be exploring how AcousticBrainz data can be utilised within Google’s BigQuery storage under the guidance of Alastair Porter.

Congratulations and good luck to all our students! We’re looking forwards to following your progress over the summer and see what you end up with. 🙂

For all the students that applied but did not get accepted: we appreciate your applications, and even if you did not make the cut this year, we hope that you will stick around and apply with us again next year when we know you better – and you know us better.

For now, let the community bonding… begin! 🙌

Deprecating MBIDs

This post is an April Fools joke. Rest assured, we have no intention of changing the MBID system that MusicBrainz currently uses.

But, like all good parody news items, there is an element of truth behind this post. The announcement of the Echo Nest API shutdown is real, and with this change you will no longer be able to use the Echo Nest IDs to look up information. This particularly hurts users of the Million Song Dataset, which maps each track to the Echo Nest ID. The new Spotify API isn’t even providing any compatibility api or ID mapping, leaving users to look up 1 million Spotify IDs in the remaining months that the the old Echo Nest api will remain available.

At MusicBrainz, we understand the importance of a stable identifier system. That’s why, 16 years ago, we picked these unwieldy-looking UUID identifiers – that have since proven to have stood the test of time, with room to continue growing. You can look up an MBID made 16 years ago, today – and it will still work another 16 years in the future.

Hello all,

Following Echo Nest’s bold announcement that Echo Nest ids are being replaced by Spotify IDs, we figured it was time to make our own ID change public as well – MBIDs were a fantastic idea 16 years ago, but let’s face it, they’re not the most beautiful thing around, so our MBIDs will now also be replaced by Spotify IDs to help with a proper mapping across tools. Anything without a Spotify mapping will simply get purged. This should greatly simplify the data we have and remove any doubt for some releases whether they exist or not – if they’re on Spotify, they clearly exist!

We would like to commend Echo Nest on their brave leadership in this, giving us the courage to move on from our ancient heritage and try new things. With the speed technologies evolve in this digital age, it can be hard to keep up with things and keep things fresh, but Echo Nest is showing the way forward, and we’re delighted to be able to follow so quickly in their path.

I hope you all will welcome this bold move by our team. We hope to have it ready by next schema change. We know we’re excited! 😀

PS. No, we will not provide a mapping between MBIDs and the new Spotify IDs. We trust our data users to be capable to set things up on their own. Happy hacking! 🙂

Say hello to Discourse!

Hello all members of the *Brainz community, I have got something in store just for you!

Some people may have noted talks and whispers about a grand and glorious move to use Discourse for various discussions related to MusicBrainz and all other MetaBrainz projects. The intention of it is to replace and unify both the now-dead mailing lists (R.I.P.) and our current forums. Guess what? The day has come at last!

The MetaBrainz Community Discourse can be found at https://community.metabrainz.org/ and is our new home for all discussions about MusicBrainz, BookBrainz, AcousticBrainz, and whatever other kind of *Brainz you want to talk about.

One of its major features is that it does not require yet another user (like the current forums, our ticket tracker, the wiki, …). When you press “Sign Up” or “Log In” it will ask you to authenticate with MusicBrainz to access some basic information. Once given permission, it will direct you back to the Discourse site and you’re logged in. (You can revoke the permission at a later point, should you need to.) No more having a dozen username/password combinations, just to participate in the community!

The site does still have some rough edges though, and various things are likely to get tweaked over the coming weeks, but today being the 1st day of (N. hemisphere) spring, I thought we should enjoy this season of “rebirth, rejuvenation, renewal, resurrection and regrowth” with this new baby of the MetaBrainz community.

A couple of people have already gone and started some discussions there, but feel free to go there yourself and start your own discussion. If you have started a discussion on the current/old forums, now is also a good time to restart/continue/move that discussion to the Discourse site as the forums will be put into read-only mode any day (posts will not be moved over).

If you don’t know where to start, start with reading the FAQ and after that, you could post to the introduction thread and introduce yourself. To get an overview about what’s going on, https://community.metabrainz.org/categories has a list of the categories currently in use and https://community.metabrainz.org/tags has a list of the tags in use. You could also just go to the front page, https://community.metabrainz.org/, and see what discussions are active right now and join in there. 🙂

I hope to see a lot of lively and friendly and constructive discussion going on there, so head over there and start making it happen. 😉

Your friendly neighbourhood community manager,
Freso

MetaBrainz & GSoC 2016

Google announced who the chosen organisations for their Google Summer of Code were earlier tonight… And MetaBrainz was amongst them for yet another year (our 10th!)!

If you’re a prospective GSoC student who is hoping to apply with MetaBrainz this year, be sure to check out our guide to getting started and our currently suggested projects – but also note that we’d love for you to come up with your own suggestion! You may also want to take a look at the application template, which contains suggestions about
some of the things you should do and know before submitting your proposal.

For those of you who are already members of our community, you’re also welcome to add ideas to the suggestions page, and please be gentle when the GSoC hopefuls come along. 🙂

That’s really all for now, but expect to hear more about GSoC and our participation over the coming months!

Wrapping up Google Code-in 2015

The Google Code-in Google Code‐in is pretty much over for this time, and we’ve had a blast in our first year with the competition in MetaBrainz with a total of 116 students completing tasks. In the end we had to pick five finalists from these, and two of these as our grand prize winners getting a trip to the Googleplex in June. It was a really, really tough decision, as we have had an amazing roster of students for our first year. In the end we picked Ohm Patel (US) and Caroline Gschwend (US) as our grand prize winners, closely followed by Stanisław Szcześniak (Poland), Divya Prakash Mittal (India), and Nurul Ariessa Norramli (Malaysia). Congratulations and thank you to all of you, as well as all our other students! We’ve been very excited to work with you and look forwards to seeing you again before, during, and after coming Google Code-ins as well! 🙂

Rayna Kanapuram MusicBrainz presentation
Indian student Rayne presenting MusicBrainz to her classmates.

In all we had 275 tasks completed during the Google Code-in. These tasks were divided among the various MetaBrainz projects as well as a few for beets. We ended up having 29 tasks done for BookBrainz, 124(!) tasks for CritiqueBrainz, 95 tasks for MusicBrainz, 1 task for Cover Art Archive, 6 tasks for MusicBrainz Picard, 3 tasks for beets, and 17 generic or MetaBrainz related tasks.

Some examples of the tasks that were done include:

Ariessa MetaBrainz infographic
Finalist Nurul Ariessa Norramli’s MetaBrainz infographic.

In all, I’m really darn happy with the outcome of this Google Code-in and how some of our finalists continue to be active on IRC and help out. Stanisław is continuing work on BookBrainz, including having started writing a Python library for BB’s API/web service, and Caroline is currently working on a new icon set for the MusicBrainz.org redesign that can currently be seen at beta.MusicBrainz.org.

Again, congratulations to our winners and finalists, and THANK YOU! to all of the students having worked on tasks for MetaBrainz. It’s really been an amazing ride and we’re definitely looking forward to our next foray into Google Code-in!

One month of Google Code-in

So today it is a month ago since the Google Code-in competition started and 18 days until it is ending. I wanted to take this opportunity to talk a bit about some of the things that have happened so far and where we’re at.

Google Code‐inSince December 7th when Google Code-in started, we have been in touch with 107 students on the Google Code-in site, of which 70 have completed at least one task and thus earned a digital certificate from Google. 11 students have so far earned themselves a t-shirt from Google by completing 3 or more tasks. The student with the highest number of completed tasks right now sits at 17 tasks, followed by one at 16 and another at 15 completed tasks. The student with the 10th most tasks completed has 3 tasks to their name.

Stanisław Szcześniak presenting about MusicBrainz
Stanisław Szcześniak, GCI student from Poland, presenting about MusicBrainz.
We have had 7 students do presentations on MusicBrainz in at least India, Romania, England, and Poland; about 50 reviews written for CritiqueBrainz with a few more in progress; a couple of MusicBrainz how to’s written for the wiki; one video tutorial made (which hasn’t been uploaded yet); a bunch of tests written for BookBrainz; updated and have had made a bunch of icons/logos in various places; a bunch of code patches and tests written for almost all our projects, as well as for beets (a 3rd party music file tagger and organiser heavily using MB data).

We have also had to report 3 students for plagiarising leading to their disqualification. 😦 However, compared to the amount of work and number of students, I think it’s a decently small number.

Overall, I am (still!) really excited about MetaBrainz finally being a part of Google Code-in, and I definitely think the lack of sleep the first week and newbie questions on IRC and on the GCI tasks are worth it. We’re getting some great stuff done, that we may not have gotten around to in any reasonable time ourselves, and we get to help all these students learn about programming, open source, open data, licenses, and a bunch of other things. I’m happy and I’m not looking forward to picking only 5 finalists and only 2 winners. There are definitely more than that I would personally like to see in both categories. 🙂

Have you had any experiences with or thoughts on our Google Code-in participation so far? Please do share them with us in the comments!

Google Code-in announcement! – and call to arms

As many of our loyal blog readers will know, the MetaBrainz Foundation has been part of the Google Summer of Code for several years. This year we’re going to expand into the realm of Google Code‐in!

Google Code‐inGoogle Code‐in, or GCI, is a program aimed at pre-university students aged 13–17. Unlike the Summer of Code (GSoC), the students will have a large number of smaller tasks to work on, instead of working on one large/huge task for the duration of the summer. It is set up as a competition and the students who do the most work will be eligible to win some Google prizes.

But what does this have to do with us? I’ll tell you what! We were accepted in this year’s Code-in! Over the last few weeks, I and a couple of other people (mostly on IRC) have been preparing for the GCI, but that does not mean we’re done yet – we can still very much use more hands. The competition starts on December 7th—that’s Monday next week! This means that we will have a flood of young people come into #musicbrainz and #metabrainz on IRC as well as possibly the forums. Please be courteous and patient; these young people will hopefully stick around Music– and MetaBrainz for long after the GCI and become full‐blown members of the community, but they need some incubation time. (Also, if you only sometimes frequent IRC, next week in particular would be a lovely time to have “all hands on deck”. Doubly so if you’re in the far eastern hemisphere as students will be coming from all different time zones and regions, and we only have a couple of East Asian and Australasian people in the IRC rooms.)

We are also continually looking for task suggestions. They can be for or from pretty much anything related to any MetaBrainz projects (we will have a number of *Brainz related tasks for beets for example), so let your creative juices flow and let us know what (small) things you would like some of these students to work on. The wikipage has some more information about what kind of tasks are acceptable. Keep in mind that they should generally be doable within 3–6 hours, but if you’re in doubt, feel free to poke one of our GCI mentors and ask.

Speaking of mentors, we could really use more mentors who are familiar with our various code bases. If you think that’s you, please, please, please poke me (Freso) on IRC and we can get things sorted. Of course, even if you don’t think you’ll be able to be a dedicated mentor, just hanging around IRC and helping students will also be a huge help and relieve the mentors somewhat to spend time on reviewing submitted work, entering additional tasks, etc.

I’m personally, and I know others are too, really excited about this. It’s a great opportunity to hopefully get a lot of low‐hanging fruit picked off as well as recruiting some new (hopefully) long‐term community members and contributors. However, it is also our first year in this program, so none of us really know what we’ve signed up for. It’s an adventure! And I hope you will be joining us. 😉

PS. If you know any pre-university students aged 13–17, tell them about the program and maybe use this as a way to lure them into the Brainzverse! 😄

Recap of the MusicBrainz Summit 15

The MusicBrainz Summit 15 participants.
From left→right (top) chirlu, reosarevok, ruaok, Freso, (bottom) Leftmost, alastairp, Gentlecat, bitmap, zas, and LordSputnik. Special guest on the laptop monitor: caller#6.

A couple of weeks ago (Oct. 30th through Nov. 1st), the MusicBrainz Summit 15 took place in Barcelona, at Rob “ruaok”‘s place. We had all of the MetaBrainz employees there, Rob/ruaok (local), Michael/bitmap (US), Nicolás/reosarevok (Spain/Estonia), Roman/Gentlecat (recently local), Laurent/zas (France), and myself, Freso (Denmark) – in addition to a bunch of other people from the community: Sean/Leftmost (US) and Ben/LordSputnik (UK), the two lead developers of BookBrainz; chirlu (Germany), long-time volunteer developer on MusicBrainz; and Alastair/alastairp (local), lead of AcousticBrainz. Between us, we represented 7 countries, 8 nationalities, and 9 languages.

Talking around the table. We managed to cover a lot of ground on the serious topics, discussing how to avoid data/MBID loss and how to version data, how to deal with labels (the entities, not the corporations…) and other unresolved style issues, how to integrate all the various *Brainz projects more and better, and a bunch of other things. The official notes for the summit is stored in a public Google Docs document. Feel free to read through and it jot down your own comments!

One of the big things was the we decided again-again-again (for the third or fourth year in a row?) to release the translations of MusicBrainz.org. But this time we actually did it! So MusicBrainz.org is now available in German, Dutch, and French (in addition to English) – go check that out if you have not done so already. 😉 At some point in the not-too-distant future™ we will also enable translating all of our documentation. Sean/Leftmost volunteered to look into options for this. Expect to hear more on that later!

MusicBrainz Style BDFL: Nicolás/reosarevok
Our Style BDFL: Nicolás a.k.a. reosarevok

We had some talk about how and why MBIDs get lost and what we can do to prevent this. As part of this discussion, we decided to make more edits autoedits for everybody. This was partly due to a wish of having a shorter queue of open edits (and there’s been a significant drop in open edits since Nov. 16!), but also very much to avoid losing MBIDs once they have been generated. More in depth discussion of the reasoning (and some of the community’s response) can be seen in the server release blog post and its comments.

We talked about a few other things like genres, reviewing the work of the style BDFL and the community manager, the future direction of the MetaBrainz Foundation, and a couple of other topics. The summit notes should contain more information on what we talked about and decided on these points.


Obviously it was not all talk and talk and talk. There was also plenty(!!) of chocolate. yeeeargh helped us by getting a lot of Ritter Sport as he apparently lives right next to their factory, and sending it along with chirlu to Barcelona. Thank you, yeeeargh! Gelato! We also managed to take in a vast amount of gelato (Italian ice cream), as there was an amazing gelato place close by Rob’s apartment. And got to walk a bit around the city of Barcelona. And have various social hanging out that only most of the time was Meta-/MusicBrainz related… but not all of it. 😉 Our system administrator, Laurent/zas, also took a bunch of pictures capturing the summit. A few of them are shown here, but you can peruse them all in the slideshow at the bottom.

Finally, a big thank you to Google and Spotify for helping to fund this meeting. It would have been a lot harder to bring all these people together from around the world without their (continued, no less!) support. Here’s to 2016 and summit 16!

This slideshow requires JavaScript.