MetaBrainz Summit 2022

The silliest, and thus best, group photo from the summit. Left to right: Aerozol, Monkey, Mayhem, Atj, lucifer (laptop), yvanzo, alastairp, Bitmap, Zas, akshaaatt

After a two-year break, in-person summits made their grand return in 2022! Contributors from all corners of the globe visited the Barcelona HQ to eat delicious local food, sample Monkey and alastairp’s beer, marvel at the architecture, try Mayhem’s cocktail robot, savour New Zealand and Irish chocolates, munch on delicious Indian snacks, and learn about the excellent Spanish culture of sleeping in. As well as, believe it or not, getting “work” done – recapping the last year, and planning, discussing, and getting excited about the future of MetaBrainz and its projects.

We also had some of the team join us via Stream; Freso (who also coordinated all the streaming and recording), reosarevok, lucifer, rdswift, and many others who popped in. Thank you for patiently waiting while we ranted and when we didn’t notice you had your hand up. lucifer – who wasn’t able to come in person because of bullshit Visa rejections – we will definitely see you next year!

A summary of the topics covered follows. The more intrepid historians among you can see full event details on the wiki page, read the minutes, look at the photo gallery, and watch the summit recordings on YouTube: Day 1, Day 2, Day 3

OAuth hack session

With everyone together, the days before the summit proper were used for some productive hack sessions. The largest of which, involving the whole team, was the planning and beginning of a single OAuth location – meaning that everyone will be sent to a single place to login, from all of our projects.

A great warmup for the summit, we also leapt forward on the project, from identifying how exactly it would work, to getting substantial amounts of code and frontend elements in place.

Project recaps

“I broke this many things this year”

To kick off the summit, after a heart-warming introduction by Mayhem, we were treated to the annual recap for each project. For the full experience, feast your eyeballs on the Day 1 summit video – or click the timestamps below. What follows is a eyeball-taster, some simplistic and soothing highlights.

State of MetaBrainz (Mayhem) (4:50)

  • Mayhem reminds the team that they’re kicking ass!
  • We’re witnessing people getting fed up with streaming and focusing on a more engaged music experience, which is exactly the type of audience we wish to cater to, so this may work out well for us.
  • In 2023 we want to expand our offerings to grow our supporters (ListenBrainz)
  • Currently staying lean to prepare for incoming inflation/recession/depression

State of ListenBrainz (lucifer) (57:10)

  • 18.4 thousand all time users
  • 595 million all time listens
  • 92.3 million new listens submitted this year (so far)
  • Stacks of updates in the last year
  • Spotify metadata cache has been a game changer

State of Infrastructure (Zas) (1:14:40)

  • We are running 47 servers, from 42 in 2019
  • 27 physical (Hetzner), 12 virtual (Hetzner), 8 active instances (Google)
  • 150 Terabytes served this year
  • 99.9% availability of core services
  • And lots of detailed server, Docker, and ansible updates, and all the speed and response time stats you can shake a stick at.

State of MusicBrainz (Bitmap) (1:37:50)

  • React conversion coming along nicely
  • Documentation improved (auto-generated schema diagrams)
  • SIR crashes fixed, schema changes, stacks of updates (genres!)
  • 1,600 active weekly editors (stable from previous years)
  • 3,401,467 releases in the database
  • 391,536 releases added since 2021, ~1,099 per day
  • 29% of releases were added by the top 25 editors
  • 51% of releases were added with some kind of importer
  • 12,607,881 genre tag votes
  • 49% of release groups have at least one genre
  • 300% increase in the ‘finnish tango’ genre (3, was 1 in 2021)

State of AcousticBrainz (alastairp) (21:01:07)

  • R.I.P. (for more on the shut down of AB, see the blog post)
  • 29,460,584 submissions
  • 1.2 million hits per day still (noting that the level of trust/accuracy of this information is very low)
  • Data dumps, with tidying of duplicates, will be released when the site goes away

State of CritiqueBrainz (alastairp) (2:17:05)

  • 10,462 total reviews
  • 443 reviews in 2022
  • Book review support!
  • General bug squashing

State of BookBrainz (Monkey) (2:55:00)

  • A graph with an arrow going up is shown, everyone applauds #business #stonks
  • Twice the amount of monthly new users compared to 2021
  • 1/7th of all editions were added in the last year
  • Small team delivering lots of updates – author credits, book ratings/reviews, unified addition form
  • Import plans for the future (e.g. Library of Congress)

State of Community (Freso) (3:25:00)

  • Continuing discussion and developments re. how MetaBrainz affects LGBTQIA2+ folks
  • New spammer and sockpuppet countermeasures
  • Room to improve moderation and reports, particularly cross-project

Again, for delicious technical details, and to hear lots of lovely contributors get thanked, watch the full recording.

Discussions

“How will we fix all the things alastairp broke”

Next (not counting sleep, great meals, and some sneaky sightseeing) we moved to open discussion of various topics. These topics were submitted by the team, topics or questions intended to guide our direction for the next year. Some of these topics were discussed in break-out groups. You can read the complete meeting minutes in the summit minutes doc.

Ratings

Ratings were added years ago, and remain prominent on MusicBrainz. The topic for discussion was: What is their future? Shall we keep them? This was one of the most popular debates at the summit, with input from the whole spectrum of rating lovers and haters. In the end it was decided to gather more input from the community before making any decisions. We invite you to regale us with tales of your useage, suggestions, and thoughts in the resulting forum thread. 5/5 discussion.

CritiqueBrainz

Similar to ratings, CritiqueBrainz has been around for a number of years now and hasn’t gained much traction. Another popular topic, with lots of discussion regarding how we could encourage community submissions, improvements that could be made, how we can integrate it more closely with the other projects. Our most prolific CB contributor, sound.and.vision, gave some invaluable feedback via the stream. Ultimately it was decided that we are happy to sunset CB as a website (without hurry), but retain its API and integrate it into our other projects. Bug fixes and maintenance will continue, but new feature development will take place in other projects.

Integrating Aerozol (design)

Aerozol (the author of this blog post, in the flesh) kicked us off by introducing himself with a little TED talk about his history and his design strengths and weaknesses. He expressed interest in being part of the ‘complete user journey’, and helping to pull MetaBrainz’ amazing work in front of the general public, while being quite polite about MeB’ current attempts in this regard. It was decided that Aerozol should focus on over-arching design roadmaps that can be used to guide project direction, and that it is the responsibility of the developers to make sure new features and updates have been reviewed by a designer (including fellow designer, Monkey).

MusicBrainz Nomenclature

Can MetaBrainz sometimes be overly-fond of technical language? To answer that, ask yourself this: Did we just use the word ’nomenclature’ instead of something simpler, like ‘words’ or ‘terms’, in this section title? Exactly. With ListenBrainz aiming for a more general audience, who expect ‘album’ instead of ‘release group’, and ‘track’ instead of ‘recording’, this was predicted to become even more of an issue. Although it was acknowledged that it’s messy and generally unsatisfying to use different terms for the same things within the same ‘MetaBrainz universe’, we decided that it was fine for ListenBrainz to use more casual language for its user-facing bits, while retaining the technical language behind the scenes/in the API.

A related issue was also discussed, regarding how we title and discuss groupings of MusicBrainz entities, which is currently inconsistent, such as “core entities”, “primary entities”, “basic entities”. No disagreements with yvanzo’s suggestions were raised, the details of which can be found in ticket MBS-12552.

ListenBrainz Roadmap

Another fun discussion (5/5 – who said ratings weren’t useful!), it was decided that for 2023 we should prioritize features that bring in new users. Suggestions revolved around integrating more features into ListenBrainz directly (for instance, integrating MusicBrainz artist and album details, CritiqueBrainz reviews and ratings), how to promote sharing (please, share your thoughts and ideas in the resulting forum thread), making landing pages more inviting for new users, and how to handle notifications.

From Project Dev to Infrastructure Maintenance

MetaBrainz shares a common ‘tech org’ problem, stemming from working in niche areas which require high levels of expertise. We have many tasks that only one or a few people know how to do. It was agreed we should have another doc sprint, which was scheduled for the third week of January (16th-20th).

Security Management / Best Practices

Possible password and identity management solutions were discussed, and how we do, and should, deal with security advisories and alerts. It was agreed that there would be a communal security review the first week of each month. There is a note that “someone” should remember to add this to the meeting agenda at the right time. Let’s see how that pans out.

Search & SOLR

Did you know that running and calibrating search engines is a difficult Artform? Indeed, a capital a Artform. Our search team discussed a future move from SOLR v7 to SOLR v9 (SOLR is MusicBrainz’ search engine). It was discussed how we could use BookBrainz as a guinea pig by moving it from ElasticSearch (the search engine BB currently runs on) to SOLR, and try finally tackle multi-entity search while we are at it. If you really like reading about ‘cores’, ‘instances’, and whatever ‘zookeeper’ is, then these are your kind of meeting minutes.

Weblate

We currently use Transifex to translate MusicBrainz to other languages (Sound interesting? Join the community translation effort!), but are planning to move to Weblate, an open-source alternative that we can self-host. Pros and cons were discussed, and it seems that Weblate can provide a number of advantages, including discussion of translation strings, and ease of implementation across all our projects. Adjusting it to allow for single-sign on will involve some work. Video tutorials and introducing the new tool to the community was put on the to-do list.

Listenbrainz Roadmap and UI/UX

When a new user comes to ListenBrainz, where are they coming from, what do they see, where are we encouraging them to click next? Can users share and invite their friends? Items discussed were specific UI improvements, how we can implement ‘calls to action’, and better sharing tools (please contribute to the community thread if you have ideas). It was acknowledged that we sometimes struggle at implementing sharing tools because the team is (largely) not made up of social media users, and that we should allow for direct sharing as well as downloading and then sharing. Spotify, Apple Music, and Last.FM users were identified as groups that we should or could focus on.

Messages and Notifications

We agreed that we should have a way of notifying users across our sites, for site-user as well as user-user interactions. There should be an ‘inbox-like’ centre for these, and adequate granular control over the notification options (send me emails, digests, no emails, etc.), and the notification UI should show notifications from all MeB projects, on every site. We discussed how a messaging system could hinder or help our anti-spam efforts, giving users a new conduit to message each other, but also giving us possible control (as opposed to the current ‘invisible’ method of letting users direct email each other). It was decided to leave messaging for now (if at all), and focus on notifications.

Year in Music

We discussed what we liked (saveable images, playlists) and what we thought could be improved (lists, design, sharing, streamlining), about last years Year in Music. We decided that this year each component needs to have a link so that it can be embedded, as well as sharing tools. We decided to publish our Year in Music in the new year, with the tentative date of Wednesday January 4th, and let Spotify go to heck with their ’not really a year yet’ December release. We decided to use their December date to put up a blog post and remind people to get their listens added or imported in time for the real YIM!

Mobile Apps

The mobile app has been making great progress, with a number of substantial updates over the last year. However it seems to be suffering an identity crisis, with people expecting it to be a tagger on the level of Picard (or not really knowing what they expect), and then leaving bad reviews. After a lot of discussion (another popular and polarising topic!) it was agreed to make a new slimmed-down ListenBrainz app to cater to the ListenBrainz audience, and leave the troubled MusicBrainz app history behind. An iOS app isn’t out of the question, but something to be left for the future. akshaaatt has beaten me to the punch with his blog post on this topic.

MusicBrainz UI/UX Roadmap

The MusicBrainz dev and design team got together to discuss how they could integrate design and a broader roadmap into the workflow. It was agreed that designers would work in Figma (a online layout/mockup design tool), and developers should decide case-by-case whether an element should be standalone or shared among sites (using the design system). We will use React-Bootstrap for shared components. As the conversion to React continues it may also be useful to pull in designers to look at UI improvements as we go. It was agreed to hold regular team meetings to make sure the roadmap gets and stays on track and to get the redesign (!) rolling.

Thank you

Revealed! Left to right: Aerozol, Monkey, Mayhem, Atj, lucifer (laptop), yvanzo, alastairp, Bitmap, Zas, akshaaatt

On behalf of everyone who attended, a huge thanks to the wonderful denizens of Barcelona and OfficeBrainz for making us all feel so welcome, and MetaBrainz for making this trip possible. See you next year!

Thank you Microsoft!

Microsoft reached out to us back in early 2018 in order to use our data in Bing — we followed the normal sort of on-boarding procedure that we use for our supporters. During one of these on-boarding calls we were asked if there was more that Microsoft could do to help us and support our mission. Soon thereafter I provided them with a list of things that would be useful to us. Sadly, the request to buy a major record label and then to give it to us to manage was turned down for being too expensive. 😦

However, Microsoft did like two items on our list and agreed to support us — they were:

1) Azure hosting credits — we’re always looking for more hosting capacity and these credits will allows us to provide virtual machines to our team and to close collaborators who are doing good work, but might be lacking the computing power to push their projects forward. This contribution is of direct benefit to our community — often times our projects contain quite a lot of data and thus have some heavy processing requirements. We’re currently using our hosting credits to do some large data set crunching and some testing for the Virtual Machine that we provide to users who wish to get up and running with MusicBrainz data quickly.

2) Sponsoring our summit — our annual team meeting and foundation summit happens at the end of each September, normally in Barcelona where we have our main office. Microsoft’s sponsorship allows us to invite more people to the event, since we have the means to cover their expenses. Our summits have traditionally been our annual forum for meeting the other team members and volunteers and to take a breather from the normal course of business. At the event we see a more human side of each other and we’re more easily able to discuss our challenges and the vision for the future.

We really appreciate our supporters who go above and beyond the normal levels of support for us — these contributions really sweeten the deal of hacking on open source software!

Thank you so much to Microsoft and everyone at Microsoft who helped move this contribution forward!

State of the Brainz: 2019 MetaBrainz Summit highlights

The 2019 MetaBrainz Summit took place on 27th–29th of September 2019 in Barcelona, Spain at the MetaBrainz HQ. The Summit is a chance for MetaBrainz staff and the community to gather and plan ahead for the next year. This report is a recap of what was discussed and what lies ahead for the community.

Continue reading “State of the Brainz: 2019 MetaBrainz Summit highlights”

MetaBrainz Summit 17

While the streets of Barcelona were filling up with the referendum conundrum, a bunch of people were spotted chattering and bantering, sometimes with pillows and colorful socks, searching for gelato.

Yes, that bunch of people would be us. 😀

Our annual MusicBrainz summit was held on September 30th–October 1st in the colorful, lively city of Barcelona. We had people (and chocolates) from nine countries: Spain, India, Germany, UK, USA, France, Estonia, Denmark, and Iceland.

Summit participants with *Brainz pillows
From left→right, top: Wieland (Mineo), Sambhav (samj1912), Sean (Leftmost Cat), Nicolás (reosarevok), Ben (LordSputnik), Jérôme (loujin), Alastair (alastairp); middle: Leo Verto, Freso, Michael (bitmap), Elizabeth (Quesito), Chhavi; bottom: Yvan (yvanzo), Rob (ruaok), Param (iliekcomputers), Suyash (ferbncode). Laurent (zas) behind the camera.


Having a majority of our team in a room with food obviously lead to lots of productive discussions. We talked about translations, recommendations engines, voting, and packaging. We also talked about SpamBrainz, user scripts being included as part of our projects, documentation, single sign-on for all Brainz, and a bunch of other things.

One of the nice things we could do this summit was to go over our user survey results. As you might remember, we had this banner on our site asking us to take part in a survey. The results gave us a good idea of our community in regards to what language they use, what Brainz project they use more, how do they come to know about us, and so on.

Summit session in progress
Summit session in progress.

We got to know what you like, but more importantly what you don’t. We heard all of you, and we are on it. We will publish a detailed report on that soon.

You cannot be in Barcelona with such a good lot of people and not end up exploring the city. The team ended up cycling on the streets of Barcelona (many times on the wrong side), climbing up to the mosaic-y Park Güell, snacking on pinchos and tapas, visiting the Pompeu Fabra University (where our AcousticBrainz project resides) and taking their daily after-lunch strolls through the Arc de Triomf.

Apart from that, some of the record-breaking points from the summit would be:

  • We had nice colorful pillows with all our kids (we mean, Brainz projects) printed on them. And summit t-shirts too.
  • The summit was live streamed on our YouTube channel, for all those who couldn’t make it. That went pretty well, with only minor technical difficulties, and it provided a good overview (literally! 🙂 ). For those who missed it (or want to rewatch it), the archived streams are available on YouTube.
  • We finally decided to improve the user experience of our projects (more on the blog about that later).
  • We worked on a new wonderful Sound Team recording while having a terrace barbeque hosted by Elizabeth.
  • More gelato was eaten than ever before. (That shouldn’t be surprising.)

We’re wrapping up the summit with this blog, but we have all the memories preserved. Find the amazing moments captured by our in-house photographer Zas in his Facebook gallery, and those moments in motion in my own video here:

Until next year,
Cheers 😀

Live streaming MB Summit 17

The MetaBrainz Summit 17 is slowly starting up, with everyone having arrived in Barcelona now, and people have already started discussing a bit in the corners of the MetaBrainz office. (As well as devouring a lot of chocolate!)

The summit officially starts tomorrow however (we’re aiming to begin at around 11 AM Barcelona time (CEST)), and while we’re having probably the most people at a summit ever, we recognise that a lot of people from the community are not able to be here for one reason or another, so we’re going to try something new tomorrow: live streaming the summit!

We’ll be live streaming on our YouTube account at https://www.youtube.com/channel/UClC89t81khDKLCVs45prLqg/live – there will be a live chat as well, which I will try to monitor as best as I can. Keep in mind that this is a first for us, so sorry in advance for the technical difficulties we will almost certainly encounter. 🙂

Recap of the MusicBrainz Summit 15

The MusicBrainz Summit 15 participants.
From left→right (top) chirlu, reosarevok, ruaok, Freso, (bottom) Leftmost, alastairp, Gentlecat, bitmap, zas, and LordSputnik. Special guest on the laptop monitor: caller#6.

A couple of weeks ago (Oct. 30th through Nov. 1st), the MusicBrainz Summit 15 took place in Barcelona, at Rob “ruaok”‘s place. We had all of the MetaBrainz employees there, Rob/ruaok (local), Michael/bitmap (US), Nicolás/reosarevok (Spain/Estonia), Roman/Gentlecat (recently local), Laurent/zas (France), and myself, Freso (Denmark) – in addition to a bunch of other people from the community: Sean/Leftmost (US) and Ben/LordSputnik (UK), the two lead developers of BookBrainz; chirlu (Germany), long-time volunteer developer on MusicBrainz; and Alastair/alastairp (local), lead of AcousticBrainz. Between us, we represented 7 countries, 8 nationalities, and 9 languages.

Talking around the table. We managed to cover a lot of ground on the serious topics, discussing how to avoid data/MBID loss and how to version data, how to deal with labels (the entities, not the corporations…) and other unresolved style issues, how to integrate all the various *Brainz projects more and better, and a bunch of other things. The official notes for the summit is stored in a public Google Docs document. Feel free to read through and it jot down your own comments!

One of the big things was the we decided again-again-again (for the third or fourth year in a row?) to release the translations of MusicBrainz.org. But this time we actually did it! So MusicBrainz.org is now available in German, Dutch, and French (in addition to English) – go check that out if you have not done so already. 😉 At some point in the not-too-distant future™ we will also enable translating all of our documentation. Sean/Leftmost volunteered to look into options for this. Expect to hear more on that later!

MusicBrainz Style BDFL: Nicolás/reosarevok
Our Style BDFL: Nicolás a.k.a. reosarevok

We had some talk about how and why MBIDs get lost and what we can do to prevent this. As part of this discussion, we decided to make more edits autoedits for everybody. This was partly due to a wish of having a shorter queue of open edits (and there’s been a significant drop in open edits since Nov. 16!), but also very much to avoid losing MBIDs once they have been generated. More in depth discussion of the reasoning (and some of the community’s response) can be seen in the server release blog post and its comments.

We talked about a few other things like genres, reviewing the work of the style BDFL and the community manager, the future direction of the MetaBrainz Foundation, and a couple of other topics. The summit notes should contain more information on what we talked about and decided on these points.


Obviously it was not all talk and talk and talk. There was also plenty(!!) of chocolate. yeeeargh helped us by getting a lot of Ritter Sport as he apparently lives right next to their factory, and sending it along with chirlu to Barcelona. Thank you, yeeeargh! Gelato! We also managed to take in a vast amount of gelato (Italian ice cream), as there was an amazing gelato place close by Rob’s apartment. And got to walk a bit around the city of Barcelona. And have various social hanging out that only most of the time was Meta-/MusicBrainz related… but not all of it. 😉 Our system administrator, Laurent/zas, also took a bunch of pictures capturing the summit. A few of them are shown here, but you can peruse them all in the slideshow at the bottom.

Finally, a big thank you to Google and Spotify for helping to fund this meeting. It would have been a lot harder to bring all these people together from around the world without their (continued, no less!) support. Here’s to 2016 and summit 16!

This slideshow requires JavaScript.

MusicBrainz Summit 11: Rotterdam

Hello everyone,

The 11th MusicBrainz summit was held this past weekend in Rotterdam, NL.

It was a very productive weekend with lots of discussions and plenty of social time. We had 17 people in attendance, 5 of which were from our sponsors/partners. The complete summit notes are now available on the wiki.

For those who didn’t attend, here are some of the things you missed out on:

  • 3 lbs of chocolate
  • banana pizza
  • stroopwafels
  • generic beer and energy drinks
  • overloading the wifi and having to rely on tethered cell phones for wifi access
  • an extremely high density of Google phones
  • 3 hours of discussion about works
  • information about BookBrainz
  • very large traffic lights

Thanks to everyone who came and to warp for being our local contact.

See you all at the next summit!