My Google Summer of Code 2022 summary

What and for whom

Organization: MetaBrainz Foundation
Project: MusicBrainz Picard
Mentors: Laurent Monin (zas) & Philipp Wolfer (phw)
Main focus: Introducing single-instance mode in Picard 3.0
GSoC website: Link

What has been done: TL;DR edition

  • Picard works in single-instance mode by default, allowing to force-spawn a new instance
  • Picard accepts not just file paths but also URLs, MBIDs and commands as command-line arguments
  • The command-line arguments are sent to the existing instance (and processed by it) if possible
  • Picard can execute commands passed by the command-line interface; e.g. save all files, show the Picard window or close the app
  • Picard can also load the commands from a text file

List of pull requests

Single-instance mode

  • Picard#2116: A big commit where the whole single-instance mode for Picard was designed and introduced (only for file paths though)
  • Picard#2135: Fixed problems with exiting the app, caused by Picard#2116
  • Picard#2130: Supported URLs (with MBIDs) and mbid:// links (documented there) can be passed with file paths via CLI to an existing (or to a new one) instance
  • Picard#2137: Supported commands (like QUIT or SHOW) can be passed via CLI to an existing instance

Picard remote commands enhancements

  • Picard#2141: REMOVE_EMPTY & REMOVE_UNCLUSTERED commands added
  • Picard#2142: LOAD command, extending the positional arguments’ functionality, added
  • Picard#2143: FROM_FILE command, executing a command pipeline from a given file, added
  • Picard#2144: CLEAR_LOGS command added
  • Picard#2145: Fixed errors with the FROM_FILE command
  • Picard#2146: WRITE_LOGS command, allowing to save Picard logs into a file, added

Code refactoring

  • Picard#2080: Code explicitly marked as deprecated got removed, my initial commit to get to know the Picard’s codebase and workflow
  • Picard#2127: Minor patch, unparsed args are now ignored as they were not used anywhere
  • Picard#2139: Refactored the whole process of passing arguments to Picard, replaced ‘%’-formatted strings with f-strings, more than one arguments can be passed correctly to a command

Other

What have I learnt during GSoC 2022

  • How to work with other people on GitHub
  • How to improve my git experience (e.g. hooks)
  • How one can handle inter-process communication, basically I have researched:
    • pipes
    • named pipes
    • sockets
    • dbus
  • How to use Windows API with Python
  • Differences between Windows and Unix pipes
  • \0 is the only character that is prohibited on both Windows & Unix in path names
  • /tmp is not the recommended way to store non-persistent app data on *nix
  • os._exit might be useful when pythonic threads get broken
  • Importing a tuple in Python is underrated. git diff gets cleaner, as one sees only the additions

Some personal thoughts

  • Python is a really decent language that helps with starting one’s programming journey but the deeper I went, the more annoyances I have encountered (that is why I ended up starting to work as a C++ dev)
  • Ultra-safety is a double-edged sword: good luck terminating Pythonic futures/threads with file operations
  • CI/CD and testing in general is as important as decent codebase
  • If one can plan their time well, flexible work hours make their work both more effective and more enjoyable
  • Python sometimes change for worse or breaks the code without any reason (e.g. they have switched from using a mode into w on pipes, ref: LINK)
  • I will not start any new personal project in Python (especially one using multi-threading, multiple processes etc.), unless forced to do so. Nu for scripting, filling the niche & exploring the functional programming, some statically-typed languages for bigger projects, games, research, etc.
  • Impostor syndrome is just an another excuse to procrastinate. Do not be scared to learn & do new things but also ask smart questions. Everyone makes mistakes but if you made it to this org, you are a good fit and have enough qualifications

Special thanks

The whole MetaBrainz community is awesome and I am glad I have become a part of it, but I would like to express my special gratitude to the people I have directly worked with in any way 🙂 (alphabetical order by github username)

GSoC’22: CritiqueBrainz reviews for BookBrainz entities

Greetings, Everyone!

I am Ansh Goyal (ansh on IRC), an undergraduate student from the Birla Institute of Technology and Science (BITS), Pilani, India. This summer, I participated in Google Summer of Code and introduced a new feature, CritiqueBrainz reviews for BookBrainz entities.

I was mentored by Alastair Porter (alastairp on IRC) and Nicolas Pelletier (monkey on IRC) during this period. This post summarizes my contributions made for this project and my experiences throughout the journey.

Proposal

Book reviews are a glimpse into a world you may or may not choose to enter. Reviews give books greater visibility and a greater chance of getting found by more readers. BookBrainz and CritiqueBrainz should enable users to rate and write reviews directly through the website using the CritiqueBrainz API.

For GSoC ’22, I made a proposal that aimed at adding CritiqueBrainz reviews for BookBrainz entities.

Community Bonding

During the community bonding period, my mentors, Alastair Porter and Nicolas Pelletier, helped me create a streamlined pathway to move along with the project. We also worked on various tickets like CB-433, CB-434, and CB-410 and added multiple features.

So we decided first to complete reviewing BookBrainz edition groups thoroughly from CritiqueBrainz as well as BookBrainz, and then extend the project for all other entities like literary work, author, and Series.

We discussed the various database and structural changes involved in the project, like adding BookBrainz Database in CritiqueBrainz, adding tables in BookBrainz DB, etc., the page designs and overall improvements.

Coding Period

The coding period starts with importing the BB database in CB to fetch the required information and perform tests. 

Now that the database was set up and ready for us to work on, it was time to write SQL queries for fetching the edition groups and all the other associated information, like identifiers and relationships. I made the code reusable to prevent duplication while fetching data for different entity types. So I opened a PR for it.

So now it was the time to allow users to review an edition group in CritiqueBrainz. For this, I made a few changes in the database, allowed BB entity types, and then added pages for their reviews in this PR. Then I worked on showing the information fetched from the BookBrainz database to the users on their entity pages.

Then to allow users to search edition groups, I worked on adding an option to search BB entities with the help of the search API already present in BB. This feature was implemented in this PR.

After adding support for Edition Groups, it was time to add support for other entity types. This expansion was very smooth because of the reusable components created by then. So I added support for Literary Works, Authors and Series. Later we discovered that the series items were not being ordered correctly, so this was fixed in #460.

During this process, my mentors and I discovered some improvements and refactoring, which were done in #445#451 and #456.

After adding support for all the entities, I added support for showing the relationships between the entities on the respective entity pages. These included showing Author-Work, Work-Edition Group, and Work-Work relationships.

CritiqueBrainz Author’s Page

While enabling CritiqueBrainz to review entities, I was also working on BookBrainz to allow users to view reviews and ratings and post them from  the entity’s page. 

So I started adding support to fetch and display reviews and ratings for edition groups which involved creating a route which would handle getting and pushing reviews to CritiqueBrainz. 

After this step, it was time to connect BookBrainz with CritiqueBrainz. This involved authentication using OAuth login. To add this feature, I first added a table ‘external_service_oauth’ in the database and then in the ORM.

Then I added routes to allow user login to CritiqueBrainz, handled the callback, and saved the tokens in the database. After that, the next thing was to allow users to post reviews from the entity page. For that, I create a modal similar to the one in ListenBrainz (to maintain consistency).

BookBrainz

After completing my project, I began working on my stretch goals and starting with unifying reviews for entities common in both BookBrainz and MusicBrainz databases. We decided that if an entity exists in both databases, we show the reviews for all the entities on the entity page (PR).

Overall Experience

I am incredibly grateful to my mentors for their constant guidance and support throughout my project. I learned a lot of technical concepts and improved the quality of my code during this journey. I had a wonderful time interacting with the amazing folks at MetaBrainz and exchanging valuable thoughts during our weekly meetings.

I would love to thank Google and the MetaBrainz Foundation for providing me with this great opportunity!

GSoC 2022: Unified Form Editor for BookBrainz

Hi, I am Shubham gupta (IRC Shubh) pursuing my bachelor’s from the National Institute of Technology, Kurushetra. This year I participated in Google Summer of Code and implemented a new editor in Bookbrainz.

In this project, I was mentored by Nicolas Pelletier (IRC monkey). The purpose of this blog is to summarize my contribution made for this project and share my experiences along the way.

Before GSoC

I joined metabrainz at the end of November’21, due to my affection for novels I instantly fell in love with BookBrainz project. I initially started working on small bug fixes and typo corrections but later shifted to work on more challenging features.

My first challenging work was to pre-fill the current entity editor with POST requests which was required for user scripts to work and also created some user scripts to help simplify the creation process.

Later I worked on to upgrade react-select and completing notification feature on BB.

Proposal

For GSoC’22, I made a proposal to work on implementing a unified form on BookBrainz.

My main motivation behind this project was to make the entity creation process more intuitive and simpler for new users. The purpose of this project is to unify all the workflows of entity creation into a simpler book interface, this abstracts away the BookBrainz specifics for users and provides them an easy interface to work with.

Though a lot has changed since the proposal, from design to implementation details, the main idea behind the project remained unchanged.

Community Bonding

During this period, I worked with my mentor to finalize the design for the editor. This included a lot of back & forth discussion but finally, we ended up picking a base design that was similar to how we choose a book: we first go through the book’s cover and back cover, then its details and inner contents.

I also discussed a new timeline with my mentor which incorporated my university classes and exams. Following this I started implementing the editor from this period itself.

Coding Period

First Phase

By this phase, I completed all the mockups and made relevant changes in design as suggested by community members.

We ended up with the following design, also later we added a summary section as a new tab to make reviewing new entities easier.

unified-form mock

I started working on the new editor routes which can support multiple entity submissions for the creation and later added support for editing as well.

Pull requests: #847#858

Since a lot of implementation was similar to existing entity routes, the main thing that was missing was to unify them into one api and make it support temporary BBIDs for new entities.

The main idea behind keeping temporary BBIDs was to allow late submission of entities, meaning new entities would only be created when the user submits the whole form. This allowed a user to undo their actions and gave them more granular control over entity creation/modification. But following this approach resulted in a lot of duplicate code which was hard to generalize due to temporary ids; this was later fixed in the second phase.

I completed working on the routes part with suitable tests and started working on the React front-end. I started by setting up a Redux store to handle multiple entity states, after some discussion with my mentor we ended up going with the state design that segregates each entity into their own states.

At the end of this phase, the editor looked something like this

First Phase Screenshot

Second Phase

During this period I continued working on the editor interface since that is the meat of the project it took most of the time of this project.

Frontend PR:#850

The challenging part of managing a large store like we had was to minimize the state updates as much as possible, since this was so crucial for performance I spent about a week reading redux articles and profiling editor state. All this paid off and resulted in blazingly fast editors (entity/unified-form) with minimal state update calls, which also benefited the existing editing pages.

The solution was to reduce the scope of a redux state by memoizing the components as much as possible and caching the results of expensive calculations which reduced component load time drastically.

After implementing all entity creation workflows, I moved to linking them, either through relationship or by some other attributes.

This linking process needs to be automatic and users don’t have to know the relationships, they should also be able to opt-out of linking specific entities with respective checkboxes.

An example of linking entities is Series-Work, where selecting a Work already adds it to a Series item.

Unified Form Series Editor

We also introduce a major change in the way we submit the entities: we now submit new entities directly to the server. This reduced the duplicate code by half as compared to before since now we don’t have to manage those temporary ids anymore. This also resulted in reducing the amount of work potentially lost when an error occurs during filling the form.

I also wrote mocha/enzyme tests for required React components. This is all for the frontend PR.

I made the follow-up PRs to improve UI and introduced bug fixes: #872#871#874

Overall Experience

I enjoyed working together with my mentor on such a large project. I learned a lot during my journey and understood the importance of different phases of software development. I realized the importance of carefully designing the application and discussing the ideas with other team members. I also got to know a lot about testing and why it is so important for large projects like this, overall the best learning experience I could ask for.

Also, the members of the MetaBrainz foundation are very supportive and help each other to resolve issues. Lastly special thanks to my mentor Nicolas Pelletier who helped me a lot during my GSoC journey. He always supported me and encouraged me even when things weren’t looking good. He is truly one of the most amazing people I’ve ever met!

MusicBrainz Server update, 2022-09-06

This release mostly consists of small bug fixes and improvements. One bigger bug fix (MBS-12497) involves an issue where it was impossible to apply any edit which would cause an artist credit with any redirects pointing to it to be removed. Sorry about your stuck edits, people! They should now pass.

A new release of MusicBrainz Docker is also available that matches this update of MusicBrainz Server. See the release notes for update instructions.

Thanks to chaban, HibiscusKazeneko, jesus2099, Mineo and sammyrayy for having reported bugs and suggested improvements. Thanks to ikerm2003 and salo.rock for updating the translations. And thanks to all others who tested the beta version!

The git tag is v-2022-09-06.

Continue reading “MusicBrainz Server update, 2022-09-06”

MusicBrainz Server update, 2022-08-22

It’s time for more fixed bugs and refactoring/maintenance tasks.

A new release of MusicBrainz Docker is also available that matches this update of MusicBrainz Server. See the release notes for update instructions.

Thanks to CatQuest and otringal for having reported bugs and suggested improvements. Thanks to hamaryns, ikerm2003 and salo.rock for updating the translations. And thanks to all others who tested the beta version!

The git tag is v-2022-08-25-hotfixes.

Continue reading “MusicBrainz Server update, 2022-08-22”

Picard 2.8.3 released

Picard 2.8.3 is a maintenance release for the recently released Picard 2.8 which fixes a potential crash in the album search dialog and provides updated translations.

The latest release is available for download on the Picard download page.

The detailed changes for this maintenance release are below. For an overview of the new features since Picard 2.7 please see our detailed release announcement for Picard 2.8 and the changes in Picard 2.8.1 and Picard 2.8.2.

Thanks a lot to everyone who gave feedback and reported issues.

Continue reading “Picard 2.8.3 released”

MusicBrainz Server update, 2022-08-08

This is a small release since we’ve been resting (and sometimes fully on holiday) for the summer. The most visible change for website users is that the “Ratings” tab for ratable entities is now a “Reviews” tab, and it also includes ratings and reviews from CritiqueBrainz. We used to only display these for release groups – now you can see them for every entity that supports reviews (artists, events, labels, places, recordings, release groups and works). We’re hoping having this reminder that things can be reviewed will encourage more users to have a say and let us know about their favourite – or less favourite – music!

Also: for people running a mirror server who updated to our May schema change and want to use tags and genres: sorry, we broke your tags table. Please check our separate blog post on how to run a script to fix it.

A new release of MusicBrainz Docker is also available that matches this update of MusicBrainz Server. See the release notes for update instructions.

Thanks to chaban and rdswift for having reported bugs and suggested improvements. Thanks to mfmeulenbelt and salo.rock for updating the translations. And thanks to all others who tested the beta version!

The git tag is v-2022-08-08.

Continue reading “MusicBrainz Server update, 2022-08-08”

Steps to fix missing genre and tag data on MusicBrainz mirror servers

If you recently updated your mirror server to the 2022-05-16 schema change release, we’re sorry to say that a bug in our upgrade script caused aggregate genre and tag data (if you had imported any) to be deleted. If you need this data, it can be re-imported from a recent dump, and we’ve written a script to help automate that.

You can safely ignore this post if

To restore the genre and tag data, follow these steps:

  1. Ensure you’ve replicated up to the most recent replication packet available. If you’re not sure, run ./admin/replication/LoadReplicationChanges. If you’re up-to-date, it should log “Replication packet … is not available.”
  2. Run git checkout production && git pull origin production.
  3. Turn off any cron jobs that update the database, including for replication.
  4. Run ./admin/sql/updates/20220720-mbs-12508.sh.
  5. Restart any cron jobs that you disabled.

You can verify that this process worked by checking the number of tags in the database: echo 'SELECT count(*) FROM tag' | ./admin/psql READWRITE. It should be over 200,000.

Sorry for the inconvenience, and let us know if you encounter any further issues.

MusicBrainz Server update, 2022-07-18

This release is very small, mainly consisting of minor improvements related to genre relationships. We have quite a few of those now, including links to Wikidata and Rate Your Music for pretty much any genre that has matching entries in those sites and links between many genres and their country of origin. We’re also fixing one bug with event setlist display that had probably been there since we introduced events – whoops!

A new release of MusicBrainz Docker is also available that matches this update of MusicBrainz Server. See the release notes for update instructions.

Thanks to chaban and sound.and.vision for having reported bugs and suggested improvements. Thanks to mfmeulenbelt, outsidecontext and salo.rock for updating the translations. And thanks to all others who tested the beta version!

The git tag is v-2022-07-18.

Continue reading “MusicBrainz Server update, 2022-07-18”

Picard 2.8.2 released

Picard 2.8.2 is a maintenance release for the recently released Picard 2.8 with fixes for reported issues and updated translations.

The latest release is available for download on the Picard download page.

The detailed changes for this maintenance release are below. For an overview of the new features since Picard 2.7 please see our detailed release announcement for Picard 2.8 and the changes in Picard 2.8.1.

Thanks a lot to everyone who gave feedback and reported issues.

Continue reading “Picard 2.8.2 released”