We’ve scheduled downtime for all our services at 15:00 UTC on October 10 (8 am PDT, 11 am EDT, 5 pm CEST). The plan is to perform some database maintenance in under an hour. Here’s a handy countdown of when we’ll begin the downtime. Thank you for your patience!
Picard 2.2.2 is available for download. Most significantly this release addresses a performance regression which affected the Windows and macOS versions of previous Picard 2 releases. It is highly recommended for Windows and macOS users to upgrade.
The complete list of changes:
- [PICARD-1606] – Crashes on opening options with broken plugin
- [PICARD-1612] – Trackpad tap is not working properly on macOS
- [PICARD-1614] – macOS: Incorrect ‘LSMinimumSystemVersion’
- [PICARD-1618] – macOS and Windows packages built without C astrcmp
- [PICARD-1621] – Lookup CD dropdown does not list additional drives
- [PICARD-1624] – Updating default CD device in options does not change default for keyboard shortcut
We have released Picard 2.2.1, which is a small bugfix release for the recently released Picard 2.2. Thanks to everybody who gave feedback and reported issues.
Picard 2.2.1 is available for download on the Picard website.
The changes in detail:
- [PICARD-1603] – Translations from picard/ui/colors.py don’t show up in Picard
- [PICARD-1604] – Windows install is not using Qt default translations
- [PICARD-1607] – Upgrading a plugin displays the dialog box multiple times
- [PICARD-1608] – “[non-album tracks]” can not directly be removed
- [PICARD-1609] – Picard About shows Qt version PyQt was build against, not actually used Qt
- [PICARD-1602] – Tests should not be included in the sdist package
The Picard Team is proud to announce the release of MusicBrainz Picard 2.2. This version provides a number of new features and bug fixes. Some of the highlights are:
- Files can be moved to sub folders without renaming the actual file (see below for details).
- Colors used for highlighting changes in files and metadata can now be configured in options.
- A new integrated media player toolbar. This feature is considered beta and is disabled by default, but you can enable the toolbar in the menu with View > Player. Please note that the file formats supported by the player depend on your operating system.
- New plugin hooks which trigger when a file was added to a release, a file was removed from a release, a file was saved and a file was loaded.
- Improved support for dropping cover art images directly from Google and Bing image search results.
- Support for ReplayGain 2.0 tags.
There are a few potentially backward incompatible changes.
- Amazon cover art moved to plugin: If you have been using the Amazon cover art you will need to install the Amazon Cover Art plugin in Options > Plugins. The functionality remains the same, we just moved it to a plugin.
- Moving files without renaming creates folder structure: Previously Picard would just drop all files into the selected target folder if “Move files” was enabled but “Rename files” was disabled. This was not very useful. Now Picard will generate the folder hierarchy according to your script.
If you want to retain the old functionality use a renaming script that does not generate any folder hierarchy (no slash or backslash characters in the script).
- The minimum supported macOS version is now macOS Sierra (10.12) or higher. If you are using macOS 10.10 or 10.11 you can continue using Picard 2.1.3.
Picard 2.2 is available for download on the Picard website.
Thanks to everybody who contributed to this release with code, translations, testing, bug reports and general feedback. This is much appreciated and we are always happy to see new contributors.
Here is the full changelog:
- [PICARD-456] – “Delete empty directories” should not delete special folders such as the desktop
- [PICARD-571] – Scripting and renaming font on macOS and Windows not monospace
- [PICARD-622] – File Browser resets horizontal scrolling on selection change
- [PICARD-765] – Refreshing a release reloads the CAA index.json file from cache, even if it changed online
- [PICARD-1025] – An empty destination directory prevents the options from being saved, but doesn’t show an error
- [PICARD-1090] – Match quality indicators are blurry
- [PICARD-1282] – ⌘W does not close Preferences window
- [PICARD-1284] – Can’t quit with preferences open
- [PICARD-1446] – Expand/collapse indicator for the release is briefly missing
- [PICARD-1483] – Can’t submit fingerprints to non-album recordings
- [PICARD-1489] – Crash on start when loading python-discid without libdiscid being available
- [PICARD-1490] – Local cover art provider fails on Windows
- [PICARD-1491] – Version check when loading Picard plugins too strict
- [PICARD-1492] – Can’t save rated tracks when it’s a FLAC file (when Metadata/Ratings is active)
- [PICARD-1493] – Crash on pre 1.0 config upgrade
- [PICARD-1497] – Saving fails when setting tags with invalid names for the underlying tagging format
- [PICARD-1499] – Picard loads embedded cover art with ID3 type “other” as sticker
- [PICARD-1501] – Double click in a cover opens it in web browser instead of an image viewer
- [PICARD-1503] – Scanning CDROM uses path containing ampersand (&)
- [PICARD-1516] – Picard fails to load MP4 without tags
- [PICARD-1517] – Functions matchedtracks and is_complete throw exception when run on cluster
- [PICARD-1522] – Crash when removing NAT recordings
- [PICARD-1527] – Can’t resize options window in 2.1.x (Mac)
- [PICARD-1529] – NAT tracks get assigned wrong cover art
- [PICARD-1533] – Attribute Qt::AA_EnableHighDpiScaling must be set before QCoreApplication is created
- [PICARD-1541] – Closing log views destroys widgets
- [PICARD-1543] – v2.1.3 crashes when selecting Preferences in the Apple menu 10.14.5
- [PICARD-1547] – Picard doesn’t warn about not updating .wav metadata
- [PICARD-1549] – Source distributions are broken on Windows
- [PICARD-1551] – “compare_to_track” method considers “score” parameter only if track has releases
- [PICARD-1556] – Default File Naming Script produces “00” track number in file name.
- [PICARD-1558] – Setting rating on a track does not apply to already matched files
- [PICARD-1566] – Cannot drag misidentified song back to the left pane
- [PICARD-1567] – Parsing track number from file name modifies original title in metadata
- [PICARD-1571] – On macOS multiple option dialogs can be opened
- [PICARD-1573] – Crash when loading release with a tag that contains only whitespace.
- [PICARD-1575] – Can’t drag and drop a picture from the Google Picture Result Page to Picard.
- [PICARD-1580] – Crash when closing options window on “Preferred Releases” page
- [PICARD-1582] – “Allow selection of multiple directories” has no effect on Linux with Gtk file chooser
- [PICARD-1584] – Crash when disabling script function providing plugin
- [PICARD-1585] – On macOS restore default options dialog opens in background
- [PICARD-1588] – Metadata box shows tags unsupported by format
- [PICARD-1591] – Error when loading Vorbis file with invalid metadata_block_picture
- [PICARD-1593] – Picard crashes on plugin install error
- [PICARD-1595] – Cursor in tag edit box always jumps to end on input
- [PICARD-1598] – Metadata box hidden when album gets updated
- [PICARD-1601] – PyPI source tarball misses some test data
- [PICARD-143] – Add a plugin hook for a file-added-to-a-track event
- [PICARD-1130] – Post save plugins
- [PICARD-1488] – Built-in media player (beta feature)
- [PICARD-1510] – Add a plugin hook for a file-removed-from-a-track event
- [PICARD-1512] – Add a plugin hook for an album-removed event
- [PICARD-1514] – Replace genre / folksonomy tag blacklist with more comprehensive list
- [PICARD-1524] – Replace hardcoded colors by user-configurable ones
- [PICARD-1560] – Add a plugin hook for a file loaded event
- [PICARD-1594] – Provide $is_video() / $is_audio scripting functions
- [PICARD-1353] – Update Travis CI to use newer Xcode
- [PICARD-1388] – Document how to uninstall local built version of picard from CLI
- [PICARD-1561] – test_file.TestPreserveTimes fails on macOS 10.14
- [PICARD-1563] – Add ‘picard.egg-info’ file to .gitignore
- [PICARD-1235] – Picard is not responding during start while CD is being inserted
- [PICARD-1361] – Add “Launch Picard” to Windows installer
- [PICARD-1400] – Remove Amazon cover art provider from Picard and place it into a plugin
- [PICARD-1468] – Localize Windows installer
- [PICARD-1485] – Picard should show the hours of long tracks
- [PICARD-1494] – Use Python3.3+ nano seconds resolution stat()/utime() to preserve times on file save
- [PICARD-1496] – Display count of Other versions available once known in album’s contextual menu
- [PICARD-1502] – qApp.setDesktopFileName (wayland app_ip)
- [PICARD-1525] – Log/History views are updated even if not actually visible
- [PICARD-1546] – Display in Others submenu is messy for albums with a lot of tracks
- [PICARD-1552] – “compare_to_release_parts” considers track count of only first medium
- [PICARD-1559] – Allow moving files to subfolders without renaming
- [PICARD-1564] – Picard code for parsing response from AcoustID servers ignores tracks
- [PICARD-1576] – Open option help context sensitive
- [PICARD-1578] – Allow dragging images from Bing image search result
- [PICARD-1579] – Dragging cover art from Google image search on Linux drops just preview image
- [PICARD-1581] – “Recursively add files and folders” is very technical and hard to understand
- [PICARD-1586] – Support for ReplayGain 2.0 tags
- [PICARD-1599] – Use fpcalc json output for more robust output parsing
It’s time for another server update! This release mostly includes small improvements to make the MusicBrainz site show data in places where it was missing and have more clear messages for the users. We have a lot of other small improvements in the pipeline which we hope to release in the next couple of updates, so if this doesn’t help with any of your pet peeves hopefully those will!
Thanks to CatQuest, chaban, danbloo, demosdemon, eey0re, ianmcorvidae, ijabz, jesus2099, Lotheric, murdos, PeterCodar, $nake, SothoTalker for having reported issues, and to every single one of you who tested the beta version and updated website localizations.
The git tag is
- [MBS-4478] – Misleading messages when adding new entities through an edit
- [MBS-10273] – Huge and weird spacing in front of the release year column on artist pages in beta
- [MBS-10320] – Don’t wrongly nag local users of MB slave server
- [MBS-10337] – ISRCs and ratings not shown when artist overview consists of recordings only
- [MBS-975] – Permanently (301) redirect from track/ to recording/
- [MBS-4161] – List blog relationship type under the External links section
- [MBS-4787] – Permanently (301) redirect http://musicbrainz.org/ns/mmd-2.0# to web service documentation
- [MBS-5049] – Show edit note syntax help in edit page too
- [MBS-10269] – Release editor: Open artist credits preview in a new window
- [MBS-10280] – When deleting user, cancel open edits from newest to oldest
- [MBS-10291] – Consider “、” as a delimiter when splitting featured artists
- [MBS-10324] – Lowercase “Takes” with guess case
- [MBS-10336] – Clarify disc.track on recording pages
- [MBS-10338] – Show ratings on artists’ recording page
We’ve recently found out about the Open Publishing Awards::
The goal of the inaugural Open Publishing Awards is to promote and celebrate a wide variety of open projects in Publishing.
All content types emanating from the Publishing sector are eligible including Open Access articles, open monographs, Open Educational Resource Materials, open data, open textbooks etc.
Open data? That’s us! We’ve got a pile of it and if you like the work we do, why not nominate us for an award?
For Starters… Who Am I?
My name is Aidan Lawford-Wickham, better known as aidanlw17 on IRC, and I’m entering my second year of undergraduate study in Engineering Science at the University of Toronto. This summer, I had the opportunity to participate in my first Google Summer of Code with the MetaBrainz Foundation. Working on the AcousticBrainz project under the mentorship of Alastair Porter (alastairp), I used previous work on measuring track to track similarity as the basis for a similarity pipeline using the entire AB database.
How Did I Get Involved?
When I started applying for GSoC, I needed to find an organization that paired a challenging learning environment with a project of personal interest. Given my own passion for listening to music, playing music, and exploring its overlap with culture, MetaBrainz quickly became my top priority. I jumped on the #metabrainz IRC channel for the first time, and I’ve been active daily ever since!
From there, the whole community welcomed me with open arms and responded thoughtfully to my questions about setting up my local development environment. I made my first pull request for AcousticBrainz, AB-387, which added the ability to include dataset and class descriptions when importing datasets as CSV files. This allowed me to work alongside my soon-to-be mentor for the first time and further acquaint myself with the acousticbrainz-server source code.
I was excited about my first PR and wanted to contribute more. Not only was this a project related to my passions, but it had already begun to teach me about technologies that I hadn’t used before. I was struck by the possibility to contribute more, and work with great people on a non-profit, open source project. I quickly decided that MetaBrainz was the only place I would apply for GSoC and began to think about proposals. I read through the previous work on recording similarity done by Philip Tovstogan, which was based upon a PostgreSQL solution with shortcomings in terms of speed. With a strong supporting background, high community interest, and my own dreams of the possibilities to come from predicting similar tracks, I created a proposal to build a similarity pipeline using Spotify’s nearest neighbours library, Annoy. The timeline and tasks shown on the full proposal were adjusted throughout the summer, but the general objectives were maintained. Looking back on the summer now, the basic requirements for the project were as such:
- Using the previous work, define metrics for measuring similarity that will translate recording features from the AB database into vectors. Compute and store these vectors for every recording in the database.
- Create an Annoy index for each of these metrics, adding the metric’s vector for each recording to the index.
- Develop methods of querying an index, such as outputting nearest neighbours (similar recordings) to a specific recording or many recordings, or finding the similarity between two recordings.
- Allow users to query the indices via an API.
- Create an evaluation that allows us to measure the success of our indices in the public eye, fine tune our parameters, and display index queries via a graphical user interface.
Community Bonding Period
After losing sleep before the announcement, and a huge sigh of relief on May 6th, I was ecstatic to get started.
There was plenty of required reading, and I familiarized myself with the different elements of building similarity into AB. After discussing with Rob (ruaok) and Alastair and cementing our decision to use Annoy as the nearest neighbours algorithm of choice, I took to reading through Annoy documentation and making a small implementation to grasp the concepts. Annoy works blazing fast, and uses small, static files – these are points that would prove advantageous for us in terms of querying indices many times, as quickly as possible. Static index files allow for them to be shared across processes and could potentially make them simple to redistribute to others in the future – a major benefit for further similarity research.
I studied Philip’s previous work, gained an understanding of the metrics he used in his thesis, and reimplemented all of his code to better grasp the concepts and use them as a basis for the summer. Much of Philip’s work was built to be easily expandable, and flexible to different types of metrics. Notably, when integrating it with a full pipeline including Annoy, priorities like speed meant that we lost some of this flexibility. I found this to be an interesting contrast between the code structure for an ongoing research purpose, and the code ready to be deployed in production on a website.
All the while, I kept a frequent dialogue with Alastair to gel as a team, clarify issues with the codebase, and further develop our plans for the pipeline. To build on my development skills, learn more about contributing guidelines and source control, and improve the site, I worked on some exciting PRs during the bonding period. Most notably, I completed AB-406 over a series of 3 PRs, which allowed us to introduce a submission offset column in the low-level table to handle multiple submissions of a single recording. This reduced the need for complexity in queries to the API, decreasing the load on the server. Additionally, I added some documentation related to contributions and created an API endpoint that would allow users to only select specific features rather than an entire low-level document for a recording – aiming at reducing server load.
Last but not least, I got really involved with the weekly meetings at MB! We have meetings every Monday on #metabrainz to give reviews of the last week, and discuss any other important community topics. I love this aspect of the community. Working remotely, it creates a strong team atmosphere and brings us all a bit closer together – even if we’re living time zones apart. During one meeting, we discussed whether or not past GSoC proposals should be available to students. What do you think? This prompted me to share my own experience with the application process at MetaBrainz and look into if/how we could improve it.
… And so it began, we dove into the first coding period.
The Key Components, a Deeper Look
Computing Similarity Metrics
Having explored the previous similarity work from Philip, I used his definitions of metric classes and focused on developing a script to compute metrics for each recording in the database incrementally. Recognizing that we would also need a method of computing metrics for a single recording on submission, I made this script as open ended as possible. After successfully computing all metrics for the first time, we went through an iterative process of altering the logic and methodology to dramatically improve its speed. Ultimately, we used a query to get the batch of low-level recordings that haven’t had similarity computations, complete with their low-level data and all high-level models. Though we revised and found bugs in this script time and time again, I’m confident in saying that with perseverance we finally got it working.
Prior to the beginning of the project I had limited experience working with SQL databases, and this objective pushed me to develop new ways to approach problems, and gave me a much deeper understanding of PostgreSQL.
Building Annoy Indices
With all that vectorized recording data from the metrics computation, nothing sounds better than adding it to an ultra-fast index built for querying nearest neighbours! Feeding the data into an index and watching it output similar recordings in milliseconds became the most satisfying feeling. The Annoy library is a platform for nearest neighbours of all sorts, and it is generally simple: define the index, add items with an identifier and a vector, built the index, save it for later use, load it up, and then use its built-in methods to query for similar items. Easy, right? The added challenge is making this interface with recordings from our database as items, and meeting our needs in terms of speed and alterability when new items are added. Annoy is built without checks in many places, and we required a custom cycle of building, loading, and saving indices to ensure they were operable for our purposes (once an index is built, new items may not be added). At this point, the index model is open to saving new indices with different parameters, which allows us to tune as we further develop the pipeline.
After wrapping the index in a class that interfaced with our needs, we added scripts to build all indices and save them, and scripts to remove indices if need be. Currently, the project has 12 indices, one for each metric in use:
- Weighted MFCCs
- Weighted GFCCs
- Onset Rate
Making API endpoints available was a high priority activity and was an exciting aspect of the project since it would allow users to interact with the data provided by a similarity pipeline. Using the index model, I created three API endpoints:
- Get the n most similar recordings to a recording specified by an (MBID, offset) combination.
- Get the n most similar recordings to a number of recordings that are specified (bulk endpoint).
- Get the distance between two recordings.
For each endpoint, a parameter indicates the metric in question, determining which index should be used. Currently, the endpoints also allow varying index parameters, such as the distance type (method of distance calculation) and number of trees used in building the index (precision increases with trees, while speed decreases).
A full explanation of the API endpoints is documented in the source code.
As I said, an index can be altered using multiple parameters that impact the build speed, query speed, and precision in finding nearest neighbours. Assessing the query results from our indices with public opinion is a top priority, since it gives us valuable data for understanding the quality of similarity predictions. With the evaluation we will be able to collect feedback from the community on a set of similar recordings – do they seem accurate, or should a recording have been more or less similar? What recording do you think is the most similar? With this sort of feedback, we can measure the success of different parameters for Annoy, eventually optimizing our results.
Moreover, this form of evaluation provides a graphical user interface to interact with similar recordings, as a user-friendly alternative to the API endpoints. Written using React, it feels snappy and fast, and I feel that it provides a pleasing display of similar recordings. At this point in the project I was glad to accept a frontend challenge which differed from the bulk of my work thus far.
Documentation and Project Links
Similarity pipeline related:
- The project set-up documentation (pull request)
- The full similarity pipeline complete with evaluation (pull request)
- Code only for computing metrics (pull request)
- Specific low-level features endpoint (pull request)
- Integrate submission offsets into low-level table (pull request 1, pull request 2, and pull request 3)
- Bulk get items with single database query (closed, unnecessary) (pull request)
- Bug tracking and git workflow documentation (pull request)
- Dataset and class descriptions for CSV import/export (pull request)
This summer allowed for us to build on previous similarity work to the point of developing a fast, full pipeline. At this point, there is still a vast amount of work to be continued on the pipeline and I am eager to see it through. In the upcoming year I plan to continue contributing to AcousticBrainz and the MetaBrainz Foundation as a whole. These are areas that I’m interested in continuing to develop for the recording similarity pipeline:
- Parameter tuning on Annoy indices
- Adding more metrics to cover other recording features
- Adding support for hybrid metrics that consider multiple features (this was started by Philip and should be integrated to provide more holistic similarity)
- Making indices available for offline use
- Creating statistics and visualizations of vectors for each metric
To say the least, this has been a highly rewarding experience. MetaBrainz is a community full of extraordinary, thoughtful, and friendly developers and enthusiasts. I will be forever thankful for this opportunity and the lessons that I gained this summer. I am so excited to meet everyone at the summit this September! I’d like to personally thank my mentor, Alastair Porter (alastairp), for his perceptive guidance, his support, his friendship, and his own contributions to the project. Thanks to Robert Kaye (ruaok) for his support, thoughts, and enthusiasm towards this project, as well as for his dedication to MetaBrainz. Thanks to Google for making this all possible – SoC is a highly unique opportunity to learn about open source software and make new connections! Cheers.