A positive outlook going forward

My next installment of MusicBrainz management changes focuses on how we should frame our discussions going forward. Currently there is a lot of animosity in our community and a lot of finger pointing — neither of these are constructive for moving forward, so I will aim to cut these short and focus on fixing rather than blaming.

I’d like to offer an analogy to start this discussion: When two people are in a personal relationship and when that relationship starts falling apart, a lot of negative feelings come up. The two people will often blame each other and be convinced that the other person is the reason for all of their troubles. If you’ve ever had an opportunity to talk to two people in a failing relationship, you’ll probably have seen that failing relationships are usually the fault of both people. I’ve yet to find a relationship that failed, solely on the actions of one person alone. Both people are involved, both people had a hand in it.

That said, I’ll step forward and say it: I am guilty. I am partially to blame for what is going on. Go ahead, feel free to blame me for the troubles we’re facing.

But, that is it. Basta! We’re not going to engage in finding every little thing that was done wrong, by whom and work hard to lay blame. That is pointless and it brings up unnecessary emotions. Instead of finding blame we’re going to find problems to our solutions and we’re going to move forward.

As part of me restructuring MusicBrainz, I’m going to be asking everyone what problems they perceive with the project right now. I will listen to the problems, catalog them and attempt to build a plan for tackling these problems in the future. However, I will insist that problems are stated without aggressive communication (e.g. passive aggressive communication) and without value judgements. If you cannot state your issue without being aggressive or disrespectful, you can count on me calling you on your behaviour. I will not address problems that are stated in an aggressive or disrespectful manner.

For instance, it is not acceptable to say: “I don’t think that anyone is going to listen to me anyway, but I think that because of Joe’s idiotic decision to not allow white space in code, all of our code is a freaking mess — this was the worst idea ever!” This statement has passive aggressive communication, it lays blame and contains a value judgement. One way to express the same concern in a constructive manner could be: “The decision to exclude whitespace from our code has created a number of difficulties for people to follow our code. We should re-consider this decision.”

This means of expressing problems, ideas and solutions allows us to focus our energy on moving forward and improving the project. It avoids painful discussions that won’t gives us much insight on moving forward. As we work to mend our community, I will be relying on these communication tools heavily. If you run afoul of these new communication guidelines, expect me to remind of you of this blog post. 🙂

Restructuring MusicBrainz’ management

In the recent past, the MusicBrainz community has become more fractured with evident tension rising between members of the community, the dev team and myself. I’ve been struggling with trying to find a good way to fix these problems and I’ve attempted making a number of changes over the past few months. Mostly with mixed or bad results, which further increased the frustrations for everyone involved.

While I was on vacation the past two weeks, I had some distance from work and at random points during this vacation a few key issues/solutions became clear to me. Over the next few weeks I will be announcing changes to how I manage this project and possibly some changes to some of our core policies to support these changes. Stay tuned on this blog for more announcements regarding this restructuring.

In the first round of changes, which I will detail in a subsequent blog posts, I would like to:

  1. Re-emphasize that we are an open source project and that we must do all of our work in public. Point 3 of our social contract states: “We won’t hide problems and policies: We will keep all MusicBrainz related discussions open for public view at all times, regardless of their content. All problems and policies related to MusicBrainz will be visible to all.” As problems in our community grew, factions hid from the public view. A lot of development work and development discussion went underground in private communication channels that had no transparency at all. Fixing this will be my most important goal moving forward.
  2. Take control of tasking the development team. Starting this monday, during our weekly development chat, I will take the lead on discussing what tasks the development team should be focusing on. I will need to catch up on a lot of happenings that I haven’t paid attention to recently. I also suspect that we will need to talk quite a bit about which tools we would like to use to manage our short, medium and long term plans. Don’t expect us to magically revamp this process on Monday — Monday will simply be the first step in what could be a long journey to improve how the MusicBrainz dev team is currently managed.

More posts are coming in the next few days!

Team change

Ian (ianmcorvidae), our senior developer on the MusicBrainz project, has decided to leave the project due to personal reasons. I’m sad to see a very skilled engineer leave our team — Ian has done a tremendous amount of great work for MusicBrainz and the MetaBrainz hosting infrastructure. Thank you for all of your hard for in the past few years, Ian!

Ian will remain on our team while he seeks a new professional position, so the change won’t be immediate. That said, now is the time to ask Ian to document whatever pieces of his work that still need documentation.

If anyone knows an experienced perl developer with experience using web technologies, Postgres and the myriad of technologies that MusicBrainz uses, please let us know! I’ll be posting an official job posting next week.

Server update, 2015-07-13

This update contains bug fixes and improvements the new tag voting interface, thanks to feedback from people during the last couple weeks. The plus/minus buttons simply add or subtract 1 from the current score (and are disabled where you’ve already done that). There’s still room for improvement, but hopefully it should be more intuitive now.

We’ve also made duplicate entity detection more immediate—the interface will show you possible duplicate artists, labels, etc. before you submit anything.

Thanks to Nikki and the MetaBrainz team for their work on today’s release. The git tag is v-2015-07-13 and the changelog is below.

Bug

  • [MBS-8322] – Editing entity (now) removes its URL relationship dates
  • [MBS-8434] – Work editor can’t seed both relationship type and target
  • [MBS-8463] – Adding a tag with upper-case letters on the tag page has funny behaviour
  • [MBS-8470] – Bulk tag submission doesn’t work if the same tag appears in different recordings

Improvement

  • [MBS-2873] – In add relationship edits, always display the URL second
  • [MBS-8331] – Move some of the duplicate entity check code to the client
  • [MBS-8464] – tag voting interface doesn’t make sense if you don’t use reddit

Server update, 2015-06-29

Today’s release progresses in finishing up the UI for last schema change. One of the changes introduced was tag voting, and as of this release you can finally upvote/downvote tags on the website. This should make it easier for people to re-use tags and to hide tags they consider wrong.

In other news, artist and place collections can be created now, too.

The rest of the release contains various bug fixes and refactorings (some without tickets, but visible in the git log).

Thanks to JesseW, legoktm, nikki, and the MetaBrainz team for their work. The git tag is v-2015-06-29 and the changelog is below.

Bug

  • [MBS-8430] – SQL exception when undoing creation of a record label: “Controller::Edit->cancel “Failed query: …” “
  • [MBS-8443] – Not possible to remove dates in the relationship editor
  • [MBS-8448] – ISE on Donation Check page

Improvement

  • [MBS-4145] – Up/down vote for tags
  • [MBS-7987] – Event series should be (auto-)ordered by date

Server update, 2015-06-15 (a day late)

This release includes a variety of bugfixes, listed below, plus a few more entities are now available for collections.

The git tag for this release is v-2015-06-15.

Bug

  • [MBS-6065] – Internal server error when double submitting an add artist edit
  • [MBS-8408] – Existing external links can be edited to have deprecated types
  • [MBS-8409] – Track parser won’t parse times for data tracks if there’s a disc ID
  • [MBS-8410] – Track parser can change the data track boundary
  • [MBS-8421] – Recording list on a work’s page has misaligned data
  • [MBS-8425] – Wikipedia extract not shown for artist with Commons link in Wikidata
  • [MBS-8431] – Statistics page sorting is broken

Improvement

  • [MBS-6737] – Improve the warning banner
  • [MBS-8031] – make mirror server’s “return to musicbrainz.org” link go to the right place

Server update, 2015-06-01

We’re back with another fortnightly release of musicbrainz-server. A small set of bugs this release, as we’re still primarily working on the bits of UI that were not finished for last fortnight’s schema change release. MBS-7489, relationship artist credits, is now finished, and (not listed below), work collections are done and ready to be used. Still remaining are collections for a variety of additional entities (eventually, everything but URLs, which would just be silly) and tag upvoting/downvoting. Thanks to all contributors and we hope you enjoy these improvements!

The git tag for this release is v-2015-06-01.

Full list of bugs:

Bug

  • [MBS-8187] – URL with open edits not removed when no relationships are left
  • [MBS-8341] – Can’t seed work attributes to /work/create: HFH: param clash for edit-work.attributes value
  • [MBS-8384] – Series-series relationships aren’t displayed
  • [MBS-8401] – “send email” broken for me (only?)

Improvement

  • [MBS-7972] – Make edit relationship edits auto-edits when the endpoints don’t change

New Feature

  • [MBS-7489] – Artist Credits for Relationships

Task

  • [MBS-8375] – Add a bunch of new sites to otherdbs
  • [MBS-8378] – Fix redirect on /tags link to docs

Schema change release, 2015-05-18 (including upgrade instructions)

Our previously mentioned schema change release is finished! Below will be upgrade instructions, including configuration updates for replication access tokens.

This release does not include UI for several of the schema change patches, which will (hopefully) happen for next release on June 1. The incomplete patches are MBS-7489 (credits for artists in relationships), MBS-4145 (tag upvote/downvote), and MBS-8004 (collections for additional entity types). These patches have had their schema change components finished, but the UI was incomplete or needed more work.

Schema Change Upgrade Instructions

These are largely as previous upgrade instructions, using the tag v-2015-05-18-schema-change. The primary difference is the inclusion of configuring an access token for replication.

  1. Make sure your REPLICATION_TYPE setting is RT_SLAVE and your DB_SCHEMA_SEQUENCE is set to 21 in lib/DBDefs.pm. If you’re running a standalone server, you can run the upgrade, but it may be easier to just import a new data dump!
  2. Ensure you’ve replicated up to the most recent replication packet available with the old schema. (if you’re not sure, run ./admin/replication/LoadReplicationChanges and see what it tells you; if you’re ready to update, it should say “Mismatched schema sequence, 21 (database) vs 22 (replication packet)”).
  3. Take down the web server running MusicBrainz, if you’re running a web server.
  4. Turn off cron jobs if you are automatically updating the database via cron jobs.
  5. Switch to the new code with git fetch origin followed by git checkout v-2015-05-18-schema-change
  6. Run ./upgrade.sh (or carton exec -Ilib -- ./upgrade.sh if you’re using carton, with very old setups).
  7. Run cpanm --installdeps --notest . to ensure your perl-based dependencies are up to date. This release adds a dependency on LWP::Protocol::https, for fetching replication packets from the new server; many systems may already have this installed, but it should be verified.
  8. Set DB_SCHEMA_SEQUENCE to 22 in lib/DBDefs.pm as instructed by the output of ./upgrade.sh
  9. Assuming you have been updating your server with replication, it will now be necessary to configure an access token:
    1. Go to https://metabrainz.org/supporters/account-type and choose your account type as applicable. If you’re an individual, non-commercial user of the data, choose “non-commercial”; if not, choose an applicable tier in the “commercial” section. If you’re not sure of the appropriate tier, make your best guess; it can be adjusted if necessary.
    2. Then, from https://metabrainz.org/profile, create an access token, which should be a 40-character random alphanumeric string provided by the site.
    3. Finally, add this token to lib/DBDefs.pm under the REPLICATION_ACCESS_TOKEN configuration option. The final configuration section should look something like sub REPLICATION_ACCESS_TOKEN { "ck3UpgwgOXhWC6SpFcd99rZOTjzfrei3gQlgZZ9z" }.
    4. Don’t reveal your access token! If you do, inadvertently, you can use the MetaBrainz site to generate a new token, invalidating the old one. (The one in the example above is one I created for myself and then invalidated — don’t get any ideas, it won’t work!)
  10. Turn cron jobs back on, if applicable.
  11. Restart the MusicBrainz web server, if applicable. It’s also recommended you restart memcached.

Finally, the list of bugs closed this release:

Bug

  • [MBS-4436] – Medium titles cannot be longer than 255 charaters

Improvement

  • [MBS-1347] – Implement aliases for release groups, releases and recordings
  • [MBS-7906] – maybe don’t show “”≠null diff. in edit pages
  • [MBS-8279] – Remove empty_artists etc. database functions

New Feature

  • [MBS-8302] – Add Live Data Feed access token support

Task

  • [MBS-8266] – Make medium titles VARCHAR NOT NULL
  • [MBS-8278] – Update DB_SCHEMA_SEQUENCE in DbDefs.pm.sample
  • [MBS-8283] – Remove DB constraint that disallows empty event names

Not included in this list but also relevant is MBS-8349, which while fixed for a previous release, in this release is also applied to old slave servers, which may help performance for some queries.

Schema upgrade downtime: Monday, 18 May, 2015 @ 17:00 UTC

Our next schema change version will be released on Monday, 18 May, 2015 around 10am PDT/1pm EDT/17:00 UTC/18:00 BST/19:00 CEST. We expect that MusicBrainz will be unavailable for 15 – 30 minutes during this time. We will put up the downtime notification on the site and tweet from @musicbrainz right before the release.

Since we’re total slackers, we still haven’t set up our backup database server since it suffered a hardware failure. This means that we won’t be able to put the site into read-only mode and will require us to do a full downtime. Hopefully for our next schema change we’ll have tackled our backlog of sysadmin duties and will have a backup DB server to make the release easier.

Sorry for any trouble this may cause you.

P.S. Look for another blog post on Sunday for details on where to get your access tokens for the Live Data Feed.

Server update, 2015-04-27

Here’s the final server update before our 2015-05-11 schema change. This posting comes late, which I apologize for, since the release coincided with switching over to our new load balancers, and I had trouble deploying things at first.

Included in this update are new “Guess feat. artists” buttons, seen on recording, release, and release group edit pages. Their purpose is to help move featured artists from titles to artist credits. To find them, look for icons with “ft.” next to the older guess-case ones.

We also now have the possibility for ordering relationships other than series ones (say, work parts), though as of this writing, they still need to be enabled as a style matter.

Thanks to chirlu, nikki, Ujjwal Wahi, and the MetaBrainz team for contributing to today’s release. The git tag is v-2015-04-27 and the complete changelog is below.

Bug

  • [MBS-8284] – Contact user form should require a verified email address
  • [MBS-8321] – Readding same information in the relationship editor gives unexpected results
  • [MBS-8349] – Unique (medium, position), (track, position) indexes no longer created on slave databases

Improvement

  • [MBS-3375] – Ability to specify sort order for sub-works that are part of an aggregate work.
  • [MBS-3799] – Show more information for releases in the inline search
  • [MBS-8332] – Make removing URLs an auto-edit for auto-editors

New Feature

  • [MBS-3388] – “Guess artist credits” – automation support for RFC-327 and moving feats etc to artist-credits

Task

  • [MBS-8344] – Remove abbreviation expanding from guess case