The 2019 MetaBrainz Summit took place on 27th–29th of September 2019 in Barcelona, Spain at the MetaBrainz HQ. The Summit is a chance for MetaBrainz staff and the community to gather and plan ahead for the next year. This report is a recap of what was discussed and what lies ahead for the community.Continue reading “State of the Brainz: 2019 MetaBrainz Summit highlights”
Note – There are no changes for Linux users, so they can safely skip this release if they want.
Given the massive feedback about the shortcomings of the Windows and macOS versions of Picard, we decided to do a minor release addressing some of the issues with our executables.
As usual, you can find the latest downloads on Picard’s Website.
The change-log is as follows –
- [PICARD-1283] – Fingerprinting not working on macOS in Picard 2.0
- [PICARD-1286] – Error creating SSL context on Windows
- [PICARD-1290] – Improve slow start up times by moving to a non single file exe
- [PICARD-1291] – Use an installer for Picard 2.x windows exe
Basically, the Windows executable is now a proper installer and some missing SSL dependencies are bundled with it.
The macOS builds also include the missing AcoustID fingerprinting binary.
The startup time for both the Windows and macOS version has been improved as well.
Have fun tagging your files!
samj1912 signing off o/
The General Data Protection Regulation is a complex EU regulation that stipulates many points for protecting private data of users on the Internet. Even though this is an EU regulation, it has a worldwide impact due to the nature of the Internet. This regulation comes into effect today, May 25, 2018 and is the reason why so many companies have sent you mail in the past few weeks about updating their privacy policies.
The MetaBrainz Foundation with its collection of projects is also affected by this regulation. We’ve been learning and adapting our sites to be compliant with the regulation – sadly this regulation isn’t entirely black and white and there is an incredible amount of room left for interpretation of these rules.
The good news is that this regulation is roughly in line with our established practices: We’ve always held private information in a high regard and applied the sort of rules to ourselves as we wish to have our own private data treated. Luckily, this makes our compliance effort considerably easier. We’ve made two significant changes to how we treat your data and also adopted terminology as used in the GDPR in order to use the same languages that many other sites are now adopting. Please keep reading to find out the exact details of what we are doing to comply.
However, we do ask for your compassion and help in our process of complying with the GDPR. As we already mentioned, the GDPR is a complex set of rules that are not fully clarified yet. We’ve taken action on the steps that are clear to us and we’re following ongoing conversations on points that are in gray zones or unclear to us. We’ve made our best initial effort on compliance and promise to keep working on it as the picture becomes more clear. If you believe that we could improve our compliance, please contact us and let us know what we can do to improve. It would also help us if you could provide concrete discussion or examples to help us understand and take action on your suggestion.
Finally, below is the link to our GDPR compliance statement, implementing the regulations as we understand them and how they affect your data in our ecosystem. Where possible, we provide links for deeper understanding, links for you to examine our relevant code and links to tickets to follow the process of improving our compliance.
As the world comes back to life after the summer break, we’re making some changes and expanding our team. First, Roman Tsukanov has decided to not renew his contract with us. During his tenure with MetaBrainz, Roman adopted and released CritiqueBrainz and also wrote our new MetaBrainz web page, which is helping us bring in new supporters. His contributions have been far from trivial — thank you for your efforts, Roman!
Due in part to the new MetaBrainz web site, we’ve got more financial support than ever, and this allows us to replace Roman with two engineers! I’m please to announce that we’re hiring two of our Summer of Code students who just completed the program:
Sambhav Kothari AKA samj1912: Sambhav started hacking on Picard earlier this year and knocked Picard out of dormancy, working towards a new release and then making Picard his Summer of Code project. He completed his project with flying colors and is working towards a major upgrade of Picard. On the MetaBrainz team he is going to look after the new search infrastructure and the maintenance and bug fixing of our Web Service in addition to hacking on Picard. A full plate, for sure!
Param Singh AKA iliekcomputers: About the same time that samj1912 arrived, Param arrived. He expressed interest in working on ListenBrainz — he too dove right in and started making improvements. ListenBrainz had quite a ways to go before he could aim to make a Summer of Code project out of it. Param and I embarked on a journey to revamp and improve the stability of ListenBrainz, which culminated in us releasing the new ListenBrainz beta a few weeks ago. Since then he’s been focusing on his Summer of Code project, which is also now complete. On the MetaBrainz team Param will be looking after ListenBrainz and also the new MetaBrainz web site.
Both Param and Sambhav will officially start working on the MetaBrainz team starting October 1, but I strongly suspect we’ll see them around and hacking on the projects as has become the norm this year.
Welcome aboard Sambhav and Param!
I’m pleased to announce that we released our first official beta version of ListenBrainz yesterday! As you may know, ListenBrainz is our project to collect, preserve and make available, user listening data similar to what Last.fm has been doing, but with open data.
In 2015 a small group of hackers gathered in London to hack on the first version of ListenBrainz alpha. We threw together a pile of new technologies and released the first version of ListenBrainz at the end of the weekend. In the end, we didn’t really like the new technologies (Cassandra, Kakfa) as both ended giving us a lot of problems that never seemed to end.
In 2016 we embarked on a journey to pick new technologies that we liked better and ended up setting on InfluxDB and RabbitMQ as backbones to our data ingestion pipeline. These tools were a good match for us, since we were already using them in production! Sadly, MetaBrainz’ move to our new hosting provider ended up sucking up any available time we had to devote to the projects, so progress was made in fits and starts.
Earlier this year Param Singh expressed interest to help with the project in hopes of joining us for a Google Summer of Code project. He started submitting a never ending stream of pull requests; slowly the project started moving forwards. Together we brought the codebase up to our current standards and integrated it into the workflow that we use for all of the MetaBrainz projects.
We proceeded to prepare the next version to be released at MetaBrainz’s new hosting facility and started a never ending series of tests. We kept pounding on the data ingestion pipeline, trying to find all of the relevant bugs and ways in which the data flow could get snagged. Finally the number of reported bugs relating to data ingestion dropped to zero and we managed to import 10M listens (a listen is a record of one song being played)!
That was our cue for promoting our pre-beta test to a full beta and unleashing it onto our production servers at our new hosting facility. Today we cleaned up the last bits of the release and we are ready for business!
What does this new release bring for you, the end users? Sadly, only a few new things, since most of the work has gone into building a stable and scalable system. We do have a few new things in this release:
- Incremental imports from Last.fm — now you don’t have to do a full import any time you wish to import your latest listens from Last.fm. The importer knows when you last did and import and will work accordingly.
- Last.fm compatible submission interface — with some system configuration changes you can submit your listens directly to ListenBrainz from any application with Last.fm support. (more info here)
- Last.fm file import — if you have an old skool Last.fm zip file with your listening history backed up, you can now import it.
- User data export — you can now download your own listens straight from the site, no waiting required.
- Adaptive rate limiting on the API — our server now uses a modern rate limiting system. For details, see our API docs.
The good news is that Param is now working on his Summer of Code project that will add a lot of graphs and other critical elements for making use of this new data set. We hope to release new features on an ongoing basis from here on out.
Most importantly, we want to publicly state that ListenBrainz is now ready for business! We don’t plan to reset the database from here on out — this is the real deal and we plan to safeguard and make this database available as soon as we can. If you have hesitated with sending your listen histories to ListenBrainz in the past, you should now feel free to send your listen information to us! If you are an author of a music player, we ask that you consider adding support for ListenBrainz in your player!
In a follow-up blog post I am going to write about how to start using ListenBrainz now — at the very least use it to back-up your Last.fm listening history!
If you find bugs with our latest release, please report them to our issue tracker. If you’re interested in this project and have questions for us, why not come and pop into our IRC channel or ask a question on our community forum?
P.S. The alpha version of ListenBrainz is still around.
P.P.S. We’ll have another cool announcement very shortly! Stay tuned!
I’m pleased to announce that Yvan Rivierre has joined the MusicBrainz development team! Yvan is not new to our community — he has been participating MusicBrainz development for some time and more recently has been attending our weekly community meeting. He’s submitted several pull requests to MusicBrainz already, and now he joins us as a full time developer.
Yvan’s nickname on all things MetaBrainz is now yvanzo, was formerly yvanz, in case you’re wondering what happened. Expect him to be around even more, helping bitmap to make improvements and changes to MusicBrainz. The MusicBrainz search infrastructure and hosting are no longer core tasks for the MusicBrainz team, leaving yvanzo and bitmap to focus solely on MusicBrainz. This brings us to a new level of dedication to our most important project and should allow us to tackle more issues faster focus new areas of improvement. (e.g. hopefully we can start making improvements in UX/UI this year!)
Welcome aboard Yvan!
In an effort to try and be even more open and transparent, the MetaBrainz Foundation’s board of directors agreed last year to publicise the minutes from its board meetings. This is the first entry of that new series. The board meets up 3–4 times a year, so don’t expect these postings to be too frequent. 🙂
These particular minutes have been approved at its following board meeting, but we will try and get future meeting minutes up prior to the next one, which means that future posts will be subject to change if the board feels something is wrong or missing in the minutes. Also note that discussion related to personnel or confidential topics have been (and will be) redacted. Furthermore we will also add more text to the notes in order to give context and background, while not altering the meaning of theses notes.
With all that said, let’s get started!
MetaBrainz Foundation Board Meeting Minutes 2016-09-15
Attendees: Robert Kaye, Nick Ashton Hart, Sophie Goossens, Rassami Hok Ljungberg, Frederik “Freso” S. Olesen, Cory Doctorow
Absent: Paul Bennun, Matthew Hawn
- 0. Call to order.
- Rob called to order at 18:15 CEST.
- Editor’s note: Unless otherwise noted, Rob will be the one speaking in the notes below.
- 1. Approval of last meeting minutes
- Moved by Rob. Rassami objects to all board members sending chocolate, but with that removed, she seconded. Unanimous. Motion carried.
- 2. Board changes
- Editor’s note: Christina, our former biz dev manager, used to be on our board of directors as both Secretary and Treasurer. Since she was no longer with us, we needed to replace her positions in the board.
- Rob moves that Christina is replaced by Rassami as Treasurer. Sophie seconds. Vote is unanimous. Motion carried.
- Rob moves to replace Christina with Frederik “Freso” Olesen, our community manager, as the new Secretary of the board. Having Freso as our secretary gives the community some insight into the board meetings. Rassami seconds. Rob suggests that Freso should take the board meeting notes and remove private/confidential points and release a version that is suitable for public consumption.
- 3. Old business:
- General update: It’s been a challenging year: Planned for a lot, had a lot of set backs. Series of perfect storms, causing extremely and challenging year for me personally, but I am finally climbing out. I’m digging out and focusing on the core issues.
- Office: Power, plumbing issues resolved. Ready for use with the hack day next week.
- Hosting: We’ve been moving services away from Digital West and to Google. At the same time we continue to get more traffic and asking people to be polite in using our service isn’t really working anymore, thus the focus on NewHost. Another perfect storm. Now have 4 servers at Hetzner in Germany, working to move in. We’ll order more servers closer to migration time. Been working all summer to move services to containers. We gain flexibility and mobility, but there is loads of work.
- Summit: We’re not going to have a summit this year, but opted for a smaller hack weekend where we iterate faster to get to NewHost.
Non paying companies: Data is wide open, and commercial customers can legally make use our data and API. Soon we will prevent them from using our API (by using API keys), but abuse is still possible. How can we deal with non-supporting companies? So far, we would put companies into the penalty box and then send a tweet to that effect. This has been our practice and I explained this to our biz dev manager, but due to other issues compounding she decided to pick this issue to start a war over. So, how should we deal with non paying companies? How far can we go in pestering companies to do the right thing?
Nick states that the heaviest offenses will be eliminated by introducing API keys. Rob notes that he still wants to bring attention to people for leech our data outside of the WS and data dumps.
Rob asks if we should try and keep track of people who download our data, but there might be some legal concerns related to (knowing if pirates download the data).
Rob to-do: ask our pro bono lawyer legal question about keeping track of people.
Rob asks: Do anyone have an objection to the “penalty box” idea?
Cory suggests to perhaps get a pro bono lawyer to send a letter to bad actors telling them they’re bad actors.
Cory also suggested that the board should be consulted before a company goes into the penalty box. Rob agrees and will write follow-up email to this point.
- 4. New business:
- Big Query: Make MusicBrainz, AcousticBrainz and ListenBrainz available on BigQuery — Google would like better and more dynamic data sets to show off their service. We get free data hosting and people will be able to query their own LB (last.fm) data on their own if they want to. Game changer. We’ve moving LB/AB, Google is adding MB. Once this is done, Google will do some publicity for this.
- Dec board meeting: London. I’ll pick a date soon.
- 5. Finances:
- Moved to new EUR capable accounting system, which took time to do, but now saves time.
- Income is increasing, this is the most amount of money we’ve had in the bank ever! > $200k!! \ø/
- Moving forward with new Biz Dev: I’m not going to replace biz dev manager right this second. I’m managing the influx of new customers, and need to focus on moving servers to new hosting facility (NewHost). Once our hosting setup is stable, I’ll work to hire more people in order to make my life more sane.
- 5. Adjourn
- 18:57 CEST