Category: musicbrainz

Picard 2.1 release

MusicBrainz Picard 2.1 is finally here. This version includes a lot of fixes and improvements over previous stable version (2.0.4). It is recommended to upgrade of course.

Most notably, it includes following changes:

  • support for MusicBrainz genres
  • new convenient script functions: $title and $delete
  • new command line option (-P, --no-plugins) allowing to skip plugin loading, which may help when a plugin is crashing Picard at start and to identify if an issue is due to a plugin or the main program
  • improve tags support, better compatibility with other applications, wider range of audio files supported
  • better stability and performance

I’d like to thank all developers, translators, testers and users who contributed to this version, kudos to Philip Wolfer (phw).

Binary packages from OSx and Windows are available from Picard Github releases page and from Picard website.

Here is the complete change log:

  • [PICARD-105] – Picard won’t load non-album tracks from fingerprints
  • [PICARD-421] – Releases in private collections are not shown as being in them
  • [PICARD-518] – Sliders without labels in “Options – Metadata – Preferred Releases”
  • [PICARD-637] – $matchedtracks is broken
  • [PICARD-875] – AIFF does not support any of the compatid3 tags
  • [PICARD-949] – Track can be placed in the incorrect spot on the release after using Scan
  • [PICARD-1013] – False file save error in specific circumstances
  • [PICARD-1060] – Collections menu not displayed correctly anymore
  • [PICARD-1112] – Cannot save tags that were previously deleted from file
  • [PICARD-1133] – Plugins list doesn’t load automatically after setting proxy
  • [PICARD-1162] – Solo vocals are tagged wrong
  • [PICARD-1219] – Picard creating empty ID3 TIPL / TMCL / IPLS frames
  • [PICARD-1245] – Set field “Grouping” doesn’t work as expected
  • [PICARD-1275] – After uninstalling a plugin Picard needs to be restarted for it to be reinstalled
  • [PICARD-1281] – Picard has wrong version string on macOS
  • [PICARD-1320] – Black text on a dark theme
  • [PICARD-1332] – Deleted tags for matched files stay deleted
  • [PICARD-1336] – MP4 reports “bpm” as unsupported tag
  • [PICARD-1339] – Removing unclustered files can be very slow
  • [PICARD-1340] – File info doesn’t display Mono / Stereo in Channels field anymore
  • [PICARD-1341] – Cluster track order misinterprets disc/track numbers
  • [PICARD-1346] – Move additional files fails if multiple patterns match
  • [PICARD-1348] – Keyboard shortcuts broken due to localization
  • [PICARD-1350] – Drag and drop on cover image box does not always work as expected
  • [PICARD-1355] – Setting or unsetting album for non-album tracks does not work
  • [PICARD-1359] – Crash with tagger integration when using DuckDuckGo Privacy Essentials
  • [PICARD-1364] – picard.exe has no version tag
  • [PICARD-1368] – Info messages are not shown on logging level Info
  • [PICARD-1369] – Crash on Python 3.7.0 opening URLs
  • [PICARD-1370] – Windows installer to add “Quick Launch” icon no longer supported on Win10
  • [PICARD-1371] – Windows installer does not warn when installing on 32 bit system
  • [PICARD-1373] – Source distributions are unusable
  • [PICARD-1374] – Picard crashes while typing a regular expression in some cases
  • [PICARD-1375] – Metadata sanitation before move-script execution fails
  • [PICARD-1376] – Error saving Ape files with tag marked for deletion that does not exist
  • [PICARD-1381] – Test results depend on execution order of tests
  • [PICARD-1397] – Do not save tags marked as unsupported
  • [PICARD-1398] – Snap package is missing locale files
  • [PICARD-1405] – Pasting formatted text into scripting window shows formatting
  • [PICARD-1410] – Loading Vorbis file with invalid rating value fails
  • [PICARD-1412] – Deleting tag counts not as important metadata change
  • [PICARD-1414] – Image errors lead to crash in info dialog
  • [PICARD-1415] – Open Containing Folder and Open with MusicPlayer does nothing for UNC paths
  • [PICARD-1418] – Display localized default dialogs and keyboard shortcut hints
  • [PICARD-1420] – Can not save wma file. TypeError: sequence item 0
  • [PICARD-1428] – Removing tags which are only in original file metadata not possible
  • [PICARD-1430] – “Authentication required” dialog does not trigger authentication
  • [PICARD-1431] – Some ID3 frames gets deleted even if the corresponding tags are shown as unchanged
  • [PICARD-1434] – Tag acoustid_id can not be removed or deleted in script
  • [PICARD-1436] – Text extraction of “title” and “label” for translation.
  • [PICARD-1437] – After reload file is being shown as changed
  • [PICARD-1438] – Message box buttons Yes/No aren’t translated
  • [PICARD-1439] – Newline character in cover art naming script causes exception on saving

New Feature

  • [PICARD-490] – Allow tagging AAC/ADTS files with APEv2 tags
  • [PICARD-1043] – Support reading & writing iTunes Classical tags
  • [PICARD-1045] – Check for new version
  • [PICARD-1268] – Support concertmaster recording relationships as performer:concertmaster
  • [PICARD-1273] – Add an option to exclude new cover art type “Raw / Unedited”
  • [PICARD-1319] – Provide cover art metadata to cover image naming script
  • [PICARD-1344] – Add $delete function
  • [PICARD-1352] – Add a command-line option to skip plugin loading
  • [PICARD-1354] – Allow using vocals and instruments as credited
  • [PICARD-1367] – Allow opening searches in browser when using search dialogs
  • [PICARD-1384] – Add AppStream data
  • [PICARD-1386] – Add $title function
  • [PICARD-1395] – Support genres from MusicBrainz
  • [PICARD-1440] – Support loading and renaming Standard MIDI Files (SMF)

Task

  • [PICARD-1333] – Run CI tests agaist oldest supported mutagen
  • [PICARD-1347] – Refactor script.py to avoid code duplication
  • [PICARD-1365] – Allow building with PyQt 5.11 or later
  • [PICARD-1442] – Support new Audio Play secondary type

Sub-task

  • [PICARD-1407] – Save originalalbum / originalartist to ASF/WMA
  • [PICARD-1408] – Save originalalbum / originalartist to APE

Improvement

  • [PICARD-664] – When dragging a recording, show the actual file name instead of the path
  • [PICARD-792] – Package a start menu tile for Windows 10 on the windows version
  • [PICARD-1039] – Use forward delete instead of delete button on macOS
  • [PICARD-1049] – Picard should use TXXX:WORK rather than TXXX:Work
  • [PICARD-1068] – Picard should use MP4 ©wrk for Work rather than generic text field
  • [PICARD-1244] – Refresh list of plugins after uninstalling or installing a local plugin
  • [PICARD-1285] – There is no Close menu item in Picard 2.0 on macOS
  • [PICARD-1313] – Refactor plugin UI
  • [PICARD-1325] – Allow disabling new version update checking for packagers
  • [PICARD-1338] – Picard should be more resilient if it gets invalid responses from servers
  • [PICARD-1358] – Use macOS style widgets in the user interface of the macOS version of Picard
  • [PICARD-1363] – AcoustId submission for matched files is impossible when musicbrainz_recordingid is unset
  • [PICARD-1366] – Show Python version in about
  • [PICARD-1379] – Port astrcmp to new Python C Unicode API
  • [PICARD-1383] – Use MCN / barcode read from disc to improve DiscId lookup
  • [PICARD-1393] – Change the application ID
  • [PICARD-1416] – Should store ID3 Artists field as TXXX:ARTISTS not TXXX:Artists
  • [PICARD-1417] – Only show plugins with compatible API version
  • [PICARD-1424] – Translate AppStream data
  • [PICARD-1425] – Support all movement tags for APE, Vorbis and MP3
  • [PICARD-1426] – Map musicbrainz_originalalbumid and musicbrainz_originalartistid to MP4 and WMA
  • [PICARD-1443] – Sort secondary release types in UI alphabetically

MusicBrainz introducing: Genres!

One of the things various people have asked MusicBrainz for time and time again has been genres. However, genres are hard to do right and they’re very much subjective—with MusicBrainz dealing almost exclusively with objective data. It’s been a recurring discussion on almost all of our summits, but a couple years ago (with some help from our friend Alastair Porter and his research colleagues at UPF), we finally came to a path forward—and recently Nicolás Tamargo (reosarevok) finally turned that path forward into code… which has now been released! You can see it in action on e.g., Nine Inch Nails’ Year Zero release group.

How does it work?

For now genres are exactly the same as (folksonomy) tags behind the scenes; some tags simply have become the chosen ones and are listed and presented as genres. The list of which tags are considered as genres is currently hardcoded, and no doubt it is missing a lot of our users’ favourite genres. We plan to expand the genre list based on your requests, so if you find a genre that is missing from it, request it by adding a style ticket with the “Genres” component.

As we mentioned above, genres are very subjective, so just like with folksonomy tags, you can upvote and downvote genres you agree or disagree with on any given entity, and you can also submit genre(s) for the entity that no one has added yet.

What about the API?

A bunch of the people asking for genres in MusicBrainz have been application developers, and this type of people are usually more interested in how to actually extract the genres from our data.

The method to request genres mirrors that of tags: you can use inc=genres to get all the genres everyone has proposed for the entity, or inc=user-genres to get all the genres you have proposed yourself (or both!). For the same release group as before, you’d want https://musicbrainz.org/ws/2/release-group/3bd76d40-7f0e-36b7-9348-91a33afee20e?inc=genres+user-genres for the XML API and https://musicbrainz.org/ws/2/release-group/3bd76d40-7f0e-36b7-9348-91a33afee20e?inc=genres+user-genres&fmt=json for the JSON API.

Since genres are tags, all the genres will continue to be served with inc=tags as before as well. As such, you can always use the tag endpoint if you would rather filter the tags by your own genre list rather than follow the MusicBrainz one, or if you want to also get other non-genre tags (maybe you want moods, or maybe you’re really interested in finding artists who perform hip hop music and were murdered – we won’t stop you!).

I use the database directly, not the API

You can parse the tag→genres from entities.json in the root of the “musicbrainz-server” repository which will give you a list of what we currently consider genres. Then you can simply compare any folksonomy tags from the %_tag tables.

Note about licensing

One thing to keep in mind for any data consumers out there is that, as per our data licensing, tags—and thus also genres—are not part of our “core (CC0-licensed) data”, but rather part of our “supplementary data” which is available under a Creative Commons Attribution-ShareAlike-NonCommercial license. Thus, if you wish to use our genre data for something commercial, you should get a commercial use license from the MetaBrainz Foundation. (Of course, if you’re going to provide a commercial product using data from MusicBrainz, you should always sign up as a supporter regardless. :)).

The future?

We are hoping to get a better coverage of genres (especially genres outside of the Western tradition, of which we have a very small amount right now) with your help! That applies both to expanding the genre list and actually applying genres to entities. For the latter, remember that everyone can downvote your genre suggestion if they don’t agree, so don’t think too much about “what genres does the world think apply to this artist/release/whatever”. Just add what you feel is right; if everyone does that we’ll get much better information. 🙂

In the near future we’re hoping to move the genre list from the code to the database (which shouldn’t mean too much for most of you, other than less waiting between a new genre being proposed for the list and it being added, but is much better for future development). Also planned is a way to indicate that several tags are the same genre (so that if you tag something as “hiphop”, “hip hop” or “hip-hop” the system will understand that’s really all the same). Further down the line, who knows! We might eventually make genres into limited entities of a sort, in order to allow linking to, say, the appropriate Wikidata/Wikipedia pages. We might do some fun stuff. Time will tell!

How five Queen songs went mainstream in totally different ways

Making graphs is easy. Making intuitive, easy-to-understand graphs? It’s harder than most people think. At the Rochester Institute of Technology, the ISTE-260 (Designing the User Experience) course teaches the language of design to IT students. For an introductory exercise in the class, students are tasked to visualize any set of data they desire. Students David Kim, Jathan Anandham, Justin W. Flory, and Scott Tinker used the MusicBrainz database to look at how five different Queen songs went mainstream in different ways. Continue reading “How five Queen songs went mainstream in totally different ways”

GSoC 2018 : Building a design system. Journey and learnings..

Hello,
I am Chhavi. I have mostly been helping around with all things design in MetaBrainz. I recently graduated from IIT Guwahati, India and started contributing to MusicBrainz after attending the summit last year, around the same time.

As a Google Summer of Code student, my project was to build a design system with React UI components for the upcoming overhaul of MusicBrainz’s website. It surely was a really interesting journey, right from when I heard about the community and I would like to share some snippets of it with you!

May 2017: I hear about Picard, and how a bunch of really cool people who meet online are building it. I was intrigued.

Around August 2017: I pop in the IRC channel #metabrainz, and after much overthinking, I drop a “Hi”. Followed was a really warm welcome by people I will soon call friends and a lot of developer-y jargon I had no clue about.

September 2017: I attend the annual MusicBrainz developer summit in Barcelona. And boy oh boy, I am now part of the family. Over the few days there, I have immense fun interacting and learning from the community.

November 2017: We set up our JIRA ticket system for design issues and start working on the mockups for the redesign. The entire community comes together on JIRA tickets and Discourse posts to talk about where we want to go with this overhaul.

January 2018: Community members encourage me to try my hand at front-end development. One is really lucky to find people, who encourage you to grow out of your comfort zone and help you cross that wall. In MetaBrainz, there is no shortage of such kind of people.

March 2018: With little confidence and lots of hopes, I apply for the Google Summer of Code programme. I start learning the ropes of development, with help of online tutorials and obviously our community. We also met for a mini-summit in Delhi to discuss ListenBrainz and spicy food.

April 2018: Hence began my full-fledged journey of learning and spending a summer of coding. It wasn’t easy, but I learned a lot in the process.

We set up the initial design system using react-bootstrap and react-storybook. I then started importing UI components into the system, followed by its documentation. I wrote up a more detailed description of the process too.

August 2018: As of now, we have the design system in place. The future plan is to continue adding components to it as well focus on having well thought contributing guidelines. I will also continue working on designing the mockups for the user interface for various entities.

Google Summer of Code was just another milestone in my journey with MetaBrainz. My time here has been a time of both personal and professional growth. I now feel more comfortable in a development environment, the ongoing chats on IRC make more sense to me and I feel less inhibited to put my thoughts out there. I completed my college, moved cities, traveled… all while having a set of these amazing people I call family.

A special shout out to Rob for keeping me going, bitmap for being ever so patient and understanding, samj1912 for introducing me to MetaBrainz, CatQuest, iliekcomputers, Suyash, Freso, reo and zas for being amazing friends through it all.

The thing I like about our community is, we had seasoned developers as well as newbies like me, all together working together to create amazing stuff. Hoping to continue being an involved and colorful part of this community,

You will obviously keep hearing from me in the coming days,
Chhavi

Instruments, part four: After Summer Special and loose ends

After 2-ish months of summer vacation I decided to do an “After Summer Special” (name as a reference to After School Specials)

By the end of summer I had been working hard… on Debussy instead of instruments! Oops!

Once September arrived I realised that I needed to get things back in order, so I set about finishing this version:

Bug

  • [INST-545] – Reverse parentage of classical/acoustic guitars

New Feature

Improvement

  • [INST-430] – akete is missing proper description (along with some improvements)
  • [INST-477] – table steel guitar alias
  • [INST-501] – Please add a couple of Dutch aliases to bullroarer
  • [INST-518] – rebec relations
  • [INST-554] – Improvements to hardingfele aliases
  • [INST-583] – add aliases to handclaps

——–

I also decided to close the tambouras (strings) version, as after much delay there wasn’t really much progress on it. I had started work on these way back in January, adding the Tamburica instruments (see Instruments part two).
However, by the time ASP came to an end it had became clear to me that I had to move on from dwelling on this, to feel like there was progress and avoid burning out.

Now, after over half the INST tickets have been closed, I had a bit more experience and I could finally finish work on this + some random stragglers left over from the “next” version.

Anyway, after all that, I decided that the thing we needed to do, was to, Get Serious

Instruments part three: The “next” branch and Even Moar Taonga pūoro

It’s time for Instruments again!
Now that I was allowed to create fix versions I set about creating such fix versions! (And also, retroactively (re)creating previous batches where these seemed logical.) This also meant creating JIRA Tickets for instruments I had already added without tickets, like the Taonga pūoro instruments.

By now, I was also starting to see the limits of the relationships between instruments we already had, so I was already thinking about what relationships could be added to improve the way we link things. As such, I created a topic to discuss some ideas I had (Reader beware! This is an early and now outdated idea-thread! (More on this in a later post!))

However, reosarevok added two more helpful link types: Artist created and Label (company) invented instrument.

next: 2017 March to May

Bug

  • [INST-442] – Alias of three-hole pipe has erroneous end date
  • [INST-465] – Typos in names, descriptions and aliases of concertinas

New Feature

Task

  • [INST-451] – Confusion in n'goni/ngɔni instruments
    • [INST-452] – Fix ngɔni relationship to banjo and picture
    • [INST-409] – Doussn'Gouni
    • [INST-453] – Fix donso ngɔni description and its relationship to kamalen ngɔni (and add relationships as improvement)
    • [INST-133] – Kamale n'goni
    • [INST-454] – Fix kamalen ngɔni description, check invention relationship (and add relationships as improvement)
    • [INST-450] – jeli ngɔni
  • [INST-476] – Conclusion of the n'goni/ngɔni instruments

Improvement

  • [INST-125] – add alias to zither (was: Harpeleik)
  • [INST-379] – Add aliases to the Melodica instrument
  • [INST-398] – add aliases to three hole pipe (was Flauta de 3 agujeros)
  • [INST-438] – Missing search hint aliases for sax instruments
  • [INST-441] – Typos in descriptions of pipe and tabor
  • [INST-443] – Add aliases to three-hole pipe and tabor

Quite a lot of tickets!

Eventually I came to recreating the Taonga pūoro tickets (previously I just added the ones in the Wikipedia page). It became clear to me that this was huge task and deserved it’s own version:

Taonga puortwo: 2017 May 9th to 25th

Updates

After this I was a bit tired and summer was finally here, so I decided to Go On Summer Vacation… so hold on for the fourth part!

GSoC 2018: SpamBrainz – Fighting spam in MusicBrainz using machine learning

Hi, I’m Leo and I spent my summer building and training SpamBrainz, our new solution to fighting spam in MusicBrainz. If you haven’t heard of SpamBrainz before it’s probably because it did not exist before this year’s Summer of Code.

For quite a while now the amount of spam in MusicBrainz has started to become a serious problem. Often this means editors are automatically created with descriptions that look not unlike the spam emails most of us get every day, promoting other websites and services.

During last year’s MetaBrainz Summit we discussed possible solutions to this and came up with the Spam Ninja system. Essentially this means that Soon™ there will be a group of editors that receive spam reports and have the ability to delete editors and entities that are nothing but spam.

Now with MusicBrainz having almost two million registered editors, could we really expect the Spam Ninjas to manually check every single one of them in addition to all the new registrations? Obviously not, and this is where SpamBrainz comes in.

SpamBrainz is a machine learning system that looks at all editors and decides whether or not it thinks they are spammers. If it thinks they are, it automatically notifies the spam ninjas who then decide whether or not SpamBrainz was correct.

What’s great about this system is that a human is guaranteed to look at any report and at no point does a computer decide that you’re a spammer and should be banned, because no one wants machines to run the world, right?

Building SpamBrainz

While most GSoC projects involve adding features to existing systems, SpamBrainz is something entirely new and I had not built anything on this scale before so I started out by doing tons of research.

When building a machine learning project you should always start by doing some good
old statistics first and trying to figure out what matters about your data and how the
system could use it. I wrote a couple Jupyter notebooks (which are great for working with data) to do this.

As I was not working for MetaBrainz at the time and had to respect our privacy policy, I wrote a script to collect the most common values of a couple different editors, anonymize them and save them to a report. Using that data I could compare all spam and non-spam editors and decide upon a set of datapoints that would be useful for my machine learning model. Yvanzo then ran these on the live database and I could happily do my data analysis without compromising user privacy.

Next I built a pretty boring Flask-based API that would allow MusicBrainz to queue up editor analysis and training. Quite a few different MetaBrainz projects use Python and need to access the MusicBrainz database so a long time ago someone wise decided to move commonly used code into a repository called brainzutils-python. All I had to do was to add some code for accessing editor data through it.

In a surprise move by ruaok I was then hired by MetaBrainz as a contractor with a yearly salary of 100g of chocolate. I probably should have negotiated what kind of chocolate but what mattered most was that I could now work with user data without breaching our privacy policy.

But before I could build my Keras model I had to decide on a final set of input features and do write code for preprocessing the data. Only then could I finally get started building and testing models.

The current SpamBrainz state of the art model is Lodbrok which actually turned out to work really well, reaching a 99% accuracy in detecting spam while only mis‐classifying 0.2% of real users as spammers. Obviously the latter won’t be a problem because after all a Spam Ninja will still check these reports.

Future outlook

Now that GSoC is over I could just disappear with all the money and leave SpamBrainz in its current state but obviously that’s not what I am planning to do.

I would like to work with zas on getting it deployed along with the Spam Ninja system, improve the code documentation and try to tackle the remaining problem that is online learning (which as it turns out, isn’t as easy as I had thought).

With spam always evolving and spammers already moving to more sophisticated methods than just using editor biographies, I’d also look into building separate models for other entities.

After all SpamBrainz is just getting started and I’m very much looking forward to continuing our journey towards reducing the spam we all have to endure on MusicBrainz and other MetaBrainz projects.