Server Update, 2012-07-09

We’ve just finished pushing out another update to the MusicBrainz web servers. This release is mainly a bug fix release, though it does see some of Alastair’s Google Summer of Code work on collections, and more of Ian’s work on translations. Also, there is finally a top editors once again.

Many thanks to Ian McEwan, Alastair Porter, Nicolás Tamargo, and the rest of the MusicBrainz team for their work on this release, and the beta team for helping us make sure everything is working nicely! Here’s what’s changed:

Bug

  • [MBS-2765] – Artist/Label guess case breaks if a different mode is set in the cookie
  • [MBS-2903] – WS/2: Can’t browse artist by work (ws/2/artist?work=MBID)
  • [MBS-3156] – Slaves should use the READONLY database settings from DBDefs.pm
  • [MBS-3904] – It is possible to vote for own edits
  • [MBS-4241] – msgids for translation should not use anything but ASCII
  • [MBS-4595] – Not all relationships are shown translated
  • [MBS-4613] – Language/script dropdowns are not translated in the release editor
  • [MBS-4731] – ISE: Edit Search for My Vote != None && Status = Open
  • [MBS-4738] – script/create_test_db.sh does not create a test database.
  • [MBS-4796] – "Other release groups" have slipped to the very bottom of the page
  • [MBS-4857] – Statistics timeline does not respond to window resizes
  • [MBS-4871] – When making a new collection, UUID in url is all uppercase
  • [MBS-4873] – Can’t go from a collection back to collections list
  • [MBS-4875] – Statistics tabs are untranslatable
  • [MBS-4878] – Dropdown lists on add artist/label/etc not translated
  • [MBS-4905] – Missing "lyrics language" in "work information" section of works tabs
  • [MBS-4926] – musicbrainz.org/release-group/{mbid} template is including an extra th
  • [MBS-4934] – robots.txt is blocking *
  • [MBS-4935] – count.recording.Nreleases statistics are calculated incorrectly and inefficiently
  • [MBS-4938] – Statistics: No. of IPIs is wrong
  • [MBS-4954] – Internal server error when checking donation status.

Improvement

  • [MBS-2387] – Better release label merging
  • [MBS-3208] – Trim leading/trailing whitespace in release editor prior to Add Missing Entities check
  • [MBS-4741] – Bring back voting icons in the edit search/edit view
  • [MBS-4855] – Split translation domains to ease translation and aid reuse of musicbrainz-server translations
  • [MBS-4856] – l() and ln() in templates should be implemented as a TT plugin, not relying on c.gettext
  • [MBS-4933] – Remove extraneous references to READWRITE in Data::Statistics

New Feature

  • [MBS-1715] – No page to view top editors

Task

  • [MBS-4845] – Update the Twitter logo
  • [MBS-4888] – Add "Trove" to "Other Databases" whitelist

Sub-task

  • [MBS-4223] – .po files with country codes don’t seem to work
  • [MBS-4224] – Interface not switching languages properly
  • [MBS-4259] – Make timeline events translatable
  • [MBS-4338] – country, language, script strings should not share identical msgid
  • [MBS-4340] – edit details template is wrongly retrieved from translated edit name

The Git tag for this release is v-2012-07-09.

Server update, 2012-06-25

We’ve just finished pushing out a small set of changes to the web servers. This is mainly a bug fix release. Thanks to hrglgrmpf, Ian McEwen, Nicolás Tamargo and the MusicBrainz team for their work on this release! Here’s what’s changed:

Bug

  • [MBS-3803] – "Edit barcodes" edit should show the previous value
  • [MBS-4341] – cron scripts don’t account for carton
  • [MBS-4513] – homepage.css included twice on main index
  • [MBS-4526] – Recordings listed on ISRC page are not sorted
  • [MBS-4580] – "Greek" as a locale of German
  • [MBS-4798] – Language statistics should be sortable
  • [MBS-4810] – CC license URLs are not fully normalized in URLCleanup.js
  • [MBS-4841] – update to new Jamendo URL scheme
  • [MBS-4893] – "No edit notes have been added." should not be on edit listing pages.

Improvement

  • [MBS-2547] – Make modbot delete unused core entities
  • [MBS-3120] – Warn when merging recordings with different ISRCs
  • [MBS-4744] – Recognise Spotify URLs
  • [MBS-4900] – Net::CoverArtArchive is not in Makefile.pl

New Feature

  • [MBS-3931] – Report: Discogs pages attached to multiple artists

Task

  • [MBS-4899] – Remove SoundUnwound links from sidebar

The Git tag for this release is v-2012-06-25.

Please Help Us Sanity Check More Removals

ocharles has just finished work on MBS-2547 which will result in empty labels and empty release groups also being deleted as part of our daily clean up jobs, just like empty artists are currently deleted. As we’ve never cleaned up empty labels or release groups in the past, there is quite a bit of data that will be deleted in the first run. Before we run this, we would like to share it with the community in case any of this data is important.

The cleanup algorithm is mostly the same as it is for artists. Labels and release groups will only be deleted if:

  • They have existed for more than 24 hours
  • They have no open edits, or open edits that show up in their edit history
  • They have no relationships
  • They have no releases

Here are the lists of labels and release groups that would be removed, if we were to run the script right now:

If you do not want a label or release group to be deleted, please add relationships, as you would with artists or works.

Server update, 2012-06-12

We’ve just finished pushing out a new set of changes to the MusicBrainz Servers. This release is mostly a bug fix, and importantly fixes a critical bug that could cause recordings to be deleted. Many thanks to Nicolás Tamargo, Ian McEwen, Wieland Hoffmann, hrglgrmpf and the MusicBrainz team for their work on this release.

Bug

  • [MBS-3862] – Sending email to an editor reveals their hidden email address when you use the "send a copy to my own email address" option.
  • [MBS-4088] – ISE and data loss (recordings) while trying to split a tracklist into 2 tracklists
  • [MBS-4398] – MySpace https URLs don’t get cleaned up properly
  • [MBS-4750] – Editing an existing artist alias to mark it as ‘primary’ doesn’t take effect.
  • [MBS-4759] – Aliases have not been migrated correctly in the last schema migration
  • [MBS-4789] – Facebook pages with certain URLs are not displayed
  • [MBS-4791] – Can’t add tags
  • [MBS-4799] – Cover art stats don’t add up
  • [MBS-4809] – CC0 1.0 license isn’t recognized by URLCleanup.js
  • [MBS-4824] – URLCleanup.js only strips whitespace from the front of strings
  • [MBS-4829] – Add Label page lies about disambiguation
  • [MBS-4834] – Detect new allmusic IDs in URLCleanup.js

Improvement

  • [MBS-3898] – Release editor should be able to deal with totaltracks in tracknumbers
  • [MBS-4302] – Remove Facebook’s "?sk=whatever" arguments during cleanup
  • [MBS-4795] – Add the Other DBs whitelisted stuff to the sidebar
  • [MBS-4814] – Add autocorrection and automatic type selection for Google Plus artist page URLs

Task

  • [MBS-4794] – Add autoselect for Other Databases

The Git tag for this release is v-2012-06-12.

Picard 1.0 released

Lukáš Lalinský announced the long-awaited release of Picard 1.0 today:

It took us a long time to get here, but I’m very happy to announce the release of Picard 1.0!

This version includes a new UI for viewing tags, some options have been rearranged and a number of bugs fixed. The file naming formats for single and various artist releases have been merged. If you were using a different format for various artist releases, Picard will upgrade the format automatically for you, but you might need to review the result. This version also comes with support for two audio fingerprinting solutions (AcoustID and AmpliFIND), but fingerprint scanning is disabled by default – you have to enable it and choose which one you want to use first.

Many thanks to Robert Kaye (ruaok), Phillipp Wolfer (outsidecontext), Michael Wiencek (bitmap), Chad Wilson (voiceinsideyou) and many many others who helped the project along the way from 2003 when it was started!

You can download the new version at:
http://musicbrainz.org/doc/MusicBrainz_Picard.

If you notice any problems, please report them at:
http://tickets.musicbrainz.org/

Complete list of changes since version 0.16:

  • New UI: Extended comparison of existing vs. MB metadata & tags (PICARD-43)
  • Merged the renaming and moving options pages
  • Removed the VA file naming format option (there is now a single format option) (PICARD-159)
  • Add %license% tag
  • Made %writer% available to tagger scripts and plugins with contents of songwriter (PICARD-21)
  • Allow two multi-valued variables to be merged in tagger scripting (PICARD-139)
  • Allow multi-valued variables to be transformed in tagger script and then set back in tags as multi-valued (PICARD-147)
  • Fix $copy not preserving multi-value variables as documented (PICARD-138)
  • Load/save free-text tags for ID3 as TXXX frames (PICARD-148)
  • Fix writing of MusicBrainz Work Id / musicbrainz_workid to tags (PICARD-88)
  • Handle mimetype for embedding cover art from EXIF jpegs (PICARD-27)
  • Change cover art box to open MusicBrainz release rather than Amazon
  • Support manual drag-and-drop of cover art onto a release via cover art box
  • Only open browser on left-click of cover art box (PICARD-190)
  • Fix Lookup in Browser (previously ‘tag lookup’) for clusters (PICARD-186)
  • Lookup in Browser will now not use MBIDs to lookup unmatched files/clusters
  • Add Date/Country to CD Lookup results dialog (PICARD-198)
  • Fix/reset album folksonomy tag counts while refreshing releases (PICARD-4)
  • Plugins actions can now create sub-menus using the MENU class attribute
  • New plugin hook register_clusterlist_action
  • Display the port Picard is listening on at bottom right status bar (PICARD-191)
  • Make album drops from right hand pane to left default to “unmatched files” again (PICARD-33)
  • Remove .DS_Store, desktop.ini, and Thumbs.db from otherwise empty directories (PICARD-75)
  • Update artist translation to use new alias features (primary flag, sort names) (PICARD-200)
  • Deleted tags aren’t indicated as changes (PICARD-165)
  • Picard log entries have inaccurate timestamp (PICARD-45)
  • Interface doesn’t allow keyboard only management (PICARD-103)
  • Added option to preserve timestamps of tagged files (PICARD-31)
  • Added keyboard shortcut to reload release (PICARD-99)
  • Medium formats weren’t listed in order in the “Other versions” menu (PICARD-91)
  • Couldn’t select multiple directories in “Add Folder” window on OS X (PICARD-74)

We've just launched our new MetaBrainz site!

In an effort to retire some old power-hungry servers, we’ve moved MusicBrainz Classic to a new server and also created a new MetaBrainz Foundation web site. The new MetaBrainz site looks like the MusicBrainz NGS site and shares a bit of the same code.

Please open a bug if you find problems with either of these two sites.

Thanks to Ollie for hacking together the metabrainz site!

Summer of Code log analysis project: May we share our data with our GSoC student?

UPDATE: This clearly going to be a major hassle, so we’ll spend the extra time coding a program that will sanitize the data before it goes into splunk.

Last week Google’s Summer of Code program started and my student Dániel Bali is ready to get busy combing through our massive logs and see what sorts of information he can mine from our logs.

We only have one minor problem — our logs contain the IP addresses of our users and some requests contain the user names of the person making the request. Removing this private information from the logs before Dániel sees them is quite a pain to do well.

I would like to propose that we:

  1. Consider Dániel part of our core team for the summer and allow him to see IP addresses and all the requests in full.
  2. Have Dániel sign a short statement stating that he will not divulge any private information.
  3. Will fail him in his GSoC project if he does divulge any private information.

If this is not acceptable to you, please speak up soon. I would like to make this happen early next week so Dániel can continue his GSoc work.

UPDATE: The final output of Dániel’s work will not contain any private information. If we end up using any private data as input, we will sanitize it and remove private information before we publish the output.

Our forums are back!

Our forums were compromised a while ago and we had to undergo massive yak shaving in order to set up a new home for our forums. Hosting many different types of software on one server makes that server hard to administer — we felt that the proper solution was to create a Virtual server host and give each type of software a new Linux instance to live in. We started with the forums, but we’re going to be moving a lot more stuff over to this virtual host in the coming weeks/months. Hopefully we can update our blog and our wiki in this process.

In any case, the forums are back online and running with the old posts, but the latest version of PunBB. In the move we lost our customized MusicBrainz theme for PunBB — if someone feels strongly about having the theme, please take a look at the PunBB docs and create a new theme. I’ll be glad to install that new theme on our server.

Thanks and sorry the forums were offline for so long!