Leaked email address incident: 2020-11-23

We’re saddened to write that we’ve let some of our users down by accidentally leaking their email addresses and birth dates via a bug in the web pages of musicbrainz.org. This caused some users to receive unwanted spam emails.

However, we would like to emphasize that no passwords, passwords hashes or any other bits of private user information other than email addresses and birth dates were leaked.

If you have never added or edited an annotation on MusicBrainz, then your email address and birth date were never leaked and you can ignore this — your data has not leaked.

What happened

About two weeks ago a MusicBrainz editor contacted us to say that their email address that was in use only at MusicBrainz had received spam. The user changed the email address to a very distinct email address in order to rule out a spammer guessing the updated email address. But it happened again, and the user received email to the unguessable email address. 

At this point we began an audit of the MusicBrainz server codebase in an attempt to find out where the leak was, patch it as soon as possible, and discover who was affected by it.

What we found

On 2019-04-26 we released a new version of the MusicBrainz server and in this version we added email addresses to the list of editor data we pass to our server to build MusicBrainz pages. The goal of this was to display them in admin-facing pages to, ironically, be able to fight spammers who were using MusicBrainz as a spamming tool. We also added the editor’s birth date, to be able to congratulate them on their birthday. Neither of these cases should have ever been a problem, since the private data should only be used on pages built and sent from our own server (where the data cannot be seen by anyone else), and any editor info sent to the users’ browser goes through a “sanitizing” process eliminating all this private information.

After some digging, we discovered that due to a bug we had overlooked in the code that stripped this data, the addresses and dates had started being sent to the browser whenever an entity page with an annotation was requested. The email address and birth date of the last person to have edited an annotation in MusicBrainz (any annotations, attached to any of our entities) was leaked on the page for the entities in question. This data was contained in a massive block of JSON data in the page source and was never shown on the web page for humans to see, which is why this issue went undetected for so long.

Who was affected

We looked at all editors who wrote any annotations that were displayed between the date the problematic code was released and the date the bug was fixed. This can mean either the annotation was written during this time period, or it was written before that but (being the latest version of the annotation for the entity) it was still displayed during this time period. This gave us a total of 17,644 editors whose data was at some point visible from the JSON block in at least one entity’s source code. We sadly do not have a way to know for sure how many of the affected were actually ever found and stored by spammers, since we attempt to block botnets as much as possible. As such, we simply have no way of knowing who was really affected by this leak — only who might have been.

What we’ve done

Once we detected the issue on November 22, we immediately put out a hotfix to all production (and beta) servers plugging the leak. The hotfix acted to sanitize the editor data by removing email addresses and birth dates from the JSON. We also deployed two additional changes that should help prevent similar issues from occurring, by avoiding sending sensitive editor data to our template renderer altogether. See all changes from the git tag v-2020-11-22-hotfix.

We are planning to improve our testing infrastructure to detect exposure of editor data — this will become a routine part of our continuous integration process. We are also going to ensure that any pull request dealing with editor data goes through a strict testing checklist.

How did spammers get these email addresses?

You might be wondering how such an obscure leak in a web page can end up in spammers finding and using your email — you’re not alone. 

Our sites are under near constant traffic from seemingly random internet bots fetching thousands of our pages in a day, with no apparent goal. All of our metadata is available for download, so why would someone download pages from us at random?

Well, we now know — web pages can contain a whole host of random data that shouldn’t be there. Email addresses, birth dates and such are just the starting point — there have been websites that have leaked credit card numbers and even login passwords, possibly compromising the integrity of user accounts.

In this case it appears that a botnet kept downloading pages from musicbrainz.org and driving the load on our servers up. We’ve been trying to block botnets ever since they’ve come into existence, but this is a laborious task that is never complete.

It appears that spammers used the botnet to scour the internet for private data such as emails to then send out lovely spam emails to all compromised users.

Summary

We would like to wholeheartedly apologize for this data leak. We take data privacy seriously and we aim to have high standards about privacy and data security. We find ourselves frustrated by the endless data leaks that happen on the Internet on a seemingly continuous basis and work hard to avoid committing these mistakes in our domain. However, we’re also human and we do make mistakes periodically. As explained above, we’re working to improve our systems and processes in order to prevent this from happening again.

We hope that you accept our most sincere apologies for this leak.

Robert Kaye, Michael Wiencek, Nicolás Tamargo and Yvan Rivierre

Picard 2.5.2 released

Picard 2.5.2 is a maintenance release, fixing some bugs and providing minor improvements to the recent 2.5.1 release. Thanks a lot to everyone who gave feedback and reported issues.

The latest release is available for download on the Picard download page.

What’s new?

Bug

  • [PICARD-1948] – ScaleFactorRoundPolicy breaks text rendering on Linux
  • [PICARD-1991] – Case-only changes to file names are not applied on case insensitive file systems on Linux
  • [PICARD-1992] – Case-only changes to file names are not applied on FAT32 and exFAT file systems on Windows 10
  • [PICARD-2001] – Directory drag & drop from file browser to cluster area broken
  • [PICARD-2004] – Metadata changes loaded asynchronously by plugins are reset if file gets matched to track
  • [PICARD-2005] – Modified fields are sometimes not correctly marked as changed when multiple files are selected
  • [PICARD-2006] – “Local files” cover provider does not detect cover files for files already present at release loading time
  • [PICARD-2012] – Loaded files not shown in UI if release MBID is a redirect
  • [PICARD-2014] – Config upgrade from Picard < 1.3.0 to version 2.4 or later fails

Improvement

  • [PICARD-1828] – Allow assigning cover art to multiple selected files
  • [PICARD-1999] – Provide binary distributions for Windows and macOS on PyPI
  • [PICARD-2007] – Disable analyze / audio fingerprinting for MIDI files

The complete list of changes of this and previous releases is available in the changelog. You can also discuss new features or usage on our forums.

MusicBrainz Server update, 2020-11-02

Right after Halloween, this new release of MusicBrainz Server tricks some bugs and treats some improvements, plus some work on the usually terrifying React conversion and updates to handle external links.

A new release of MusicBrainz Docker is also available that matches this update of MusicBrainz Server. See the release notes for update instructions.

Thanks to chaban, darwinx0r, kellnerd, hibiscuskazaneko, jesus2099, lotheric, snartal, and tularion for having reported bugs and suggested improvements. Thanks to grafi_tt, mfmeulenbelt, salorock, and shepard for updating the translations. And thanks to all others who tested the beta version!

The git tag is v-2020-11-02.

Bug

  • [MBS-6666] – Artist credits not renamed from artist edit page unless the artist name is changed
  • [MBS-10281] – Improper encoding of ISE pages
  • [MBS-10829] – Indexed recording search fails to find recording with no length
  • [MBS-11160] – Internal server error pages display empty stack traces
  • [MBS-11161] – Internal server error page sometimes not returned when an error occurs
  • [MBS-11186] – Inconsistent username font-weight for edit owner
  • [MBS-11194] – TypeError: Cannot read property ‘linkTypeID’ of undefined (part 2)
  • [MBS-11204] – ISE: Validation failed for \’Int\’ with value undef

Improvement

  • [MBS-7219] – Only display “Show only standalone recordings instead” when there are standalone recordings to display
  • [MBS-11158] – Document URL link_type integers for release editor seeding
  • [MBS-11177] – Do not show useless “Description:” label in entity type doc boxes
  • [MBS-11185] – Add “is not” operator for relationship type in edit search
  • [MBS-11192] – Add voting-icon for Approved
  • [MBS-11197] – Add validation for Mainly Norfolk links
  • [MBS-11199] – Update 7digital.com URL cleanup

React Conversion Task

  • [MBS-11195] – Convert the artist credit renamer to React

Other Task

  • [MBS-11182] – Remove LyricWiki links from the sidebar
  • [MBS-11189] – Remove PureVolume links from sidebar
  • [MBS-11196] – Add saisaibatake.ame-zaiku.com to “other databases” for instruments
  • [MBS-11200] – Add works to VGMdb autocleanup

Picard 2.5.1 released

Picard 2.5.1 is a maintenance release, fixing some bugs and providing minor improvements to the recent 2.5 release. Thanks a lot to everyone who gave feedback and reported issues.

The latest release is available for download on the Picard download page.

What’s new?

Bugfixes

  • [PICARD-1987] – Special variables %_multiartist%, %_pregap%, %_datatrack% and %_totalalbumtracks% missing after files got matched to a track
  • [PICARD-1988] – Aborts if directory scan finds a directory which cannot be read
  • [PICARD-1989] – Auto-sizing does not work right on first column
  • [PICARD-1990] – Case-only changes to file names are not applied on Windows if running with Python >= 3.8

Improvements

  • [PICARD-1985] – Support chorus master recording relationships as “performer:chorus master” instead of second conductor
  • [PICARD-1995] – Add command line parameter –no-player to disable media player
  • [PICARD-1997] – Reduce performance impact of fingerprint status column

The complete list of changes of this and previous releases is available in the changelog. You can also discuss new features or usage on our forums.

Picard 2.5 release

Picard 2.5 is now available.

Thanks a lot to everybody who contributed to this release with code, translations, bug reports and general feedback.

What’s new?

This release contains few fixes and improvements over previous beta version. A complete list of changes since the beta release is available below.
New packages can be downloaded from Picard website Downloads section.

As usual, please report any issue through our bug tracker.

Bug

  • [PICARD-214] – Take case insensitive filesystems into account when renaming files
  • [PICARD-1972] – Times missing in bottom panel
  • [PICARD-1973] – Multi-value tags getting flattened when files get matched to tracks
  • [PICARD-1974] – Picard crashes when using the same tag name more than once in “Tags from File Names…”
  • [PICARD-1975] – Tags from file names does not properly set hidden tags
  • [PICARD-1976] – Cover art providers do not handle URLs with query arguments correctly
  • [PICARD-1979] – Comment tag in MP4 not saved correctly
  • [PICARD-1982] – Coverart not saving properly

Improvement

  • [PICARD-1978] – Add keyboard shortcut for Tags From Filenames and allow to place it in toolbar

The whole Picard team would like to thank everyone involved in this project, whether they are developpers, users, translators, documentation contributors.

Since some time the whole Picard documentation is better than ever, thanks, mainly, to the fantastic job rdswift did. With the help of the community we should be able to provide translations for this documentation in upcoming future (tests were done recently). If you want to help, read Contributing page.

You can also discuss new features or usage on our forums.

MusicBrainz Server update, 2020-10-19

Today’s MusicBrainz Server brings a new data report, a continued conversion to React, some bugfixes and small improvements, but also tests refactoring.

Meanwhile, the search server has been updated twice in a row to fix bugs in JSON output mostly with MB Solr 3.2 (release notes) and MB Solr 3.3 (release notes), including the MusicBrainz API breaking change announced last month.

A new release of MusicBrainz Docker is also available that matches this update of MusicBrainz Server. See the release notes for update instructions.

Thanks to amCap1712 for fixing bugs in MB Solr, and to loujine for contributing code with yet a new data report. Thanks to Avamander, bonchiver_, chaban, draconx, eloise_freya, GTF1982, hawke, hibiscuskazeneko, jesus2099, jrv, kellnerd, Kid Devine, Psychoadept, selflessself, and wcw1966 for having reported bugs and suggested improvements. Thanks to kellnerd, mfmeulenbelt, and salorock for updating the translations. And thanks to all others who tested the beta version!

The git tag is v-2020-10-19.

Bug

  • [MBS-10221] – Track Parser not filling in artists
  • [MBS-11149] – Misspelling of the word “misspellings”
  • [MBS-11150] – Add CD-TOC wrongly defaults to last listed artist when none selected
  • [MBS-11156] – Track parser unsets “Various Artists” track artist credits
  • [MBS-11162] – Work type description bubble starts as default even if type is selected
  • [MBS-11174] – Editor profile added entities: Missing Add release edit type 216
  • [MBS-11176] – Heading of the release group section in the external links sidebar has disappeared

Improvement

  • [MBS-5225] – Allow showing tracklists everywhere when attaching/viewing discIDs
  • [MBS-7256] – Add “Expand all mediums” option to the release page
  • [MBS-8725] – Allow mediums to have an unknown tracklist
  • [MBS-11115] – Show detailed information when attaching disc IDs
  • [MBS-11139] – Use HTTPS for display on Library of Congress links
  • [MBS-11163] – Show type descriptions when editing entities
  • [MBS-11165] – Update the VK logo used in the sidebar
  • [MBS-11167] – Normalize vk.com links to HTTPS
  • [MBS-11173] – When deleting users, change their No and Yes votes on pending edits to Abstain

New Feature

  • [MBS-11117] – Report for mediums with very long durations from discID

React Conversion Task

  • [MBS-11141] – Convert Edit Relationship edit to React
  • [MBS-11152] – Convert entity/ratings page to React

Other Task

  • [MBS-11148] – Remove Google Play links from the sidebar

Picard 2.5 Beta 1

Picard 2.5 Beta 1 is now available. This is a pre-release to gather final feedback on the changes before the final 2.5 release.

Thanks a lot to everybody who contributed to this release with code, translations, bug reports and general feedback.

What’s new?

This release fixes some possible crashes, makes Picard able to run on the new macOS 11, provides several small UI improvements, allows using file tags and variables in tagger script, and more. See below for a full list of changes.

Bugfixes

  • [PICARD-1858] – MusicBrainz Picard does not respond on macOS 11 Big Sur Beta
  • [PICARD-1882] – Scripting text not well readable with dark theme on KDE
  • [PICARD-1888] – Returning tracks to cluster uses matched release rather than what’s in the files
  • [PICARD-1932] – Failed AcoustID submission shows as successful
  • [PICARD-1939] – Crash when “Remove” button is hit without picking a file first
  • [PICARD-1941] – Unchanged multi-value tags on tracks show up as changed
  • [PICARD-1954] – Right-clicking on album with “could not load album” message crashes
  • [PICARD-1956] – It is possible to have the same file multiple times inside the same cluster
  • [PICARD-1961] – Capitalization for non-standardized instruments
  • [PICARD-1963] – Possible stack overflow when loading files
  • [PICARD-1964] – Scripting documentation does not support RTL languages
  • [PICARD-1969] – Browser integration port changes without saving options
  • [PICARD-1971] – Tags from file names dialog does not restore window size

New Features

  • [PICARD-259] – Make file-specific variables and metadata available to tagger script
  • [PICARD-534] – Support SOCKS proxy
  • [PICARD-1908] – Allow loading release group URLs / MBIDs
  • [PICARD-1942] – Display tracklist in Info dialog for loaded releases
  • [PICARD-1946] – Map “vocal arranger” rel to arranger tag

Improvements

  • [PICARD-1390] – Apply a network timeout to avoid network requests hanging indefinitely
  • [PICARD-1782] – Allow locking table headers to prevent accidental resorting
  • [PICARD-1879] – When dragging tracks onto a release add those tracks sequentially
  • [PICARD-1906] – Clarify uninstall message in Windows installer
  • [PICARD-1936] – Rename the “Whitelist” cover art provider to “Allowed Cover Art URLs”
  • [PICARD-1937] – Add context menu entry to cover art box to browse for local file
  • [PICARD-1938] – Add context menu option to load files / folders from file browser
  • [PICARD-1951] – Avoid complete hiding of metadata box or file panes
  • [PICARD-1952] – Allow using native Qt styles on Linux
  • [PICARD-1955] – Use built-in search by default for new installs
  • [PICARD-1957] – Load files in file browser on double click
  • [PICARD-1958] – macOS: Offer link to Applications folder in disk image
  • [PICARD-1959] – In disc ID dialog rename “Lookup in Browser” to “Submit disc ID”
  • [PICARD-1960] – Allow adding new items in list editor views using Insert key
  • [PICARD-1965] – Allow opening fingerprinting options in AcoustID missing API key dialog

Tasks

Download

Picard 2.5 beta 1 is available for download from the download page.

Helping out

The easiest way to help us getting a great Picard 2.5 release is using and testing this release candidate. Please report bugs on the Picard issue tracker and provide feedback in the community forums.

Please also help translate Picard. There have been many changes to the user interface and existing translations need to be updated for the final 2.4 release. Translating is easy and can be done online: Head over to MusicBrainz’s translation page on Transifex and click on “Help Translate MusicBrainz”.
Once you have registered an account on Transifex you can start translating. For Picard the primary resource to translate is “picard“, but there is also the “picard_appstream” resource which is used for providing descriptions for various Linux software-center applications.

If you are a software developer you are very welcomed to provide fixes and features. Picard is free software and the source code is available on GitHub. See Developing on the Picard website to get started.

MusicBrainz Server update, 2020-10-05

As we just returned from the virtual MusicBrainz Summit 20, here comes a mostly maintenance release that fixes a bunch of bugs (among which many are about localization) and provides a few handy improvements. It even features a new report showing releases having the same barcode but currently in different release groups, so if you feel like it, do try and help us look into those!

A new release of MusicBrainz Docker is also available that matches this update of MusicBrainz Server. See the release notes for update instructions.

Special thanks to the DistriNet Research Group for making and responsibly sharing a very detailed audit of our OAuth service that has been very helpful with releasing major security improvements in the previous server update!

Thanks to chaban, fabe56, hibiscuskazeneko, humhumxx, jesus2099, lotheric, mfmeulenbelt, nikki, otringal, psychoadept, and wcw1966 for having reported bugs and suggested improvements. Thanks to dimpole, listmycds, mfmeulenbelt, peter9811 for updating the translations. And thanks to all others who tested the beta version!

The git tag is v-2020-10-05.

Bug

  • [MBS-4118] – Date/Number.toLocaleString always uses English
  • [MBS-10956] – Ratings are not loaded in some pages
  • [MBS-11094] – Edit error message appears (and prevents update) unrelated to current edits
  • [MBS-11107] – span.name-variation class is missing on some relationship credits
  • [MBS-11114] – Can’t set track length to 0:00
  • [MBS-11140] – Client side language bundles are not built in production

Improvement

  • [MBS-3116] – Warn when merging works with different ISWCs
  • [MBS-8650] – Accept IPIs with length of 5 characters or longer
  • [MBS-11033] – Update the Apple Music logo used in the sidebar
  • [MBS-11036] – Don’t allow to add new release when moving CD TOC
  • [MBS-11104] – Missing “Credited As” column in Label Relationships tab
  • [MBS-11121] – Update iTunes/Apple Music URL cleanup
  • [MBS-11127] – Don’t return unrelated recording-work relationships within work-level-rels from /ws/2/release
  • [MBS-11136] – Support HTML in admin/attribute descriptions
  • [MBS-11137] – Show basic release info on hover on frontpage’s release additions

New Feature

  • [MBS-6322] – New Report: Releases with identical barcodes in different Release Groups

React Conversion Task

  • [MBS-10991] – Convert user/subscriptions pages to React

MusicBrainz Server update, 2020-09-21

React conversion tasks are conspicuously absent from today’s release, but that’s just because we needed to take some time to get it all working with the recent refactoring. This new server update mainly brings strong security improvements for the OAuth service. It also comes with a fair amount of smaller bugfixes and improvements. The most noticeable of these probably are the added details to the merge recordings’ form and the statistics by entity type on editors’ profile pages.

Announcement for MusicBrainz API users: A small but breaking change will be deployed on October 19th (in one month from now), to fix the JSON formatting of release packaging in search results (SEARCH-579).

A new release of MusicBrainz Docker is also available that matches this update of MusicBrainz Server. See the release notes for update instructions.

Thanks to kellnerd and loujine for contributing code. Thanks to calculator.ftvb, chaban, hibiscuskazeneko, jesus2099, kellnerd, lalinksy, psychoadept, rdswift, and spitzwegerich for having reported bugs and suggested improvements. Thanks to jesus2099, kellnerd, mfmeulenbelt, outsidecontext, and salorock for updating the translations. And thanks to all others who tested the beta version!

The git tag is v-2020-09-21.

Bug

  • [MBS-10880] – Series automatic ordering (without numbers) fails for new release group
  • [MBS-11065] – Smart link blocks affecting legitimate links
  • [MBS-11069] – Diff highlighting not visible for certain display resolutions
  • [MBS-11098] – Big Cartel URLs are denied for labels
  • [MBS-11101] – Series relationships not showing for work series

Improvement

  • [MBS-2768] – Display AcoustIDs, Annotation and any other useful info when merging recordings
  • [MBS-7473] – Adding a new discid: Allow to specify the target by its releaseid
  • [MBS-11017] – Normalize IMSLP URLs to HTTPS and add validation
  • [MBS-11058] – Tighten security of OAuth service
  • [MBS-11061] – Don’t allow MusicBrainz URLs in relationships
  • [MBS-11062] – Link basic how-tos from the front page
  • [MBS-11086] – Add icon for tabs with errors in release editor
  • [MBS-11109] – Block further more smart links
  • [MBS-11119] – Set a Content-Security-Policy header on account/admin related forms

New Feature

  • [MBS-7485] – OAuth token revokation through API
  • [MBS-9769] – Show entities added statistics on editor profile page
  • [MBS-10835] – Disallow creating new accounts with an e-mail already in use
  • [MBS-11097] – Support PKCE (Proof Key for Code Exchange) by OAuth clients

Task

  • [MBS-10921] – Clear editing history of unrelated recording-of relationship edits

MusicBrainz Server update, 2020-09-07

Beyond the restless conversion to React of edits’ display, this new release of MusicBrainz Server features the sidebar display of recordings’ acoustic information automatically computed by AcousticBrainz, and brings a handful of more discreet improvements and fixed bugs.

A new release of MusicBrainz Docker is also available that matches this update of MusicBrainz Server. See the release notes for update instructions.

Thanks to loujine for contributing the code to display AcousticBrainz data. Thanks to chaban, draconx, hawke, scotia, and yindesu for having reported bugs and suggested improvements. Thanks to kellnerd, mfmeulenbelt, salorock, and speardog for updating the translations (de, el, fr, it, nl). And thanks to all others who tested the beta version!

The git tag is v-2020-09-09-hotfixes.

Bug

  • [MBS-11000] – ISE when trying to display “Edit release group” edit
  • [MBS-11039] – Entity present twice in a series with different numbers appears with the same number in the Series page
  • [MBS-11074] – JavaScript is broken in IE11
  • [MBS-11075] – core-js polyfills are included twice in JavaScript bundles
  • [MBS-11076] – Size of data-context attributes used for React hydration bloats page size
  • [MBS-11077] – CritiqueBrainz reviews have disappeared
  • [MBS-11078] – Internal Server Error: undef error – TypeError: Cannot read property ‘names’ of undefined (hotfixed)
  • [MBS-11081] – Pregap info not shown in Add medium edits (hotfixed)
  • [MBS-11085] – Cannot edit 8cm CD release with disc ID (hotfixed)
  • [MBS-11089] – Homepage and blog favicons gone from sidebar (hotfixed)

Improvement

  • [MBS-7409] – Make “cannot attach discid” more obvious when format can’t have discID
  • [MBS-10916] – Show AcousticBrainz info in the Recording sidebar
  • [MBS-10941] – Rename XML Web Service into “MusicBrainz API” (JSON/XML)
  • [MBS-11014] – Provide context to “This relationship already exists” on relationship editor
  • [MBS-11041] – Make ModBot leave a note on autoremoval edits
  • [MBS-11073] – Drop lodash in favor of native JavaScript methods

React Conversion Task

  • [MBS-10972] – Convert Add Instrument edit to React
  • [MBS-10986] – Convert edit error templates to React
  • [MBS-11032] – Convert Add medium edit to React
  • [MBS-11034] – Convert Remove medium edit to React
  • [MBS-11048] – Convert Edit Area edit to React
  • [MBS-11050] – Convert Edit URL edit to React
  • [MBS-11051] – Convert Edit Artist Credit edit to React
  • [MBS-11052] – Convert Edit Relationship Attribute edit to React

Other Task

  • [MBS-11043] – Add offiziellecharts.de to the otherdbs whitelist
  • [MBS-11044] – Drop any references to no longer existing TOBEDELETED edit status