Category: Development

New autumn schema change date: 17 November, 2014

Due to two conflicting summits in (our own and the GSoC Summit) around our usual schema change release date, we’ve decided to move the autumn schema change to 17 November, 2014. This will ensure that our developers are properly rested before attempting a hard task such as a schema change.

We hope this won’t cause too much trouble for everyone downstream.

Server update, 2014-06-23

It’s that exciting time again: another release of bug fixes and improvements. User-visible changes include a menu showing recently-used entities in autocomplete lookups (previously only available in the relationship editor), plus the display of other identities/legal names on artist pages. Thanks go to the MetaBrainz team and chirlu and nikki for their hard work on this release!

The git tag is v-2014-06-23.

Here’s the complete changelog:

Bug

  • [MBS-3537] – Add a unique constraint on medium (release, position)
  • [MBS-7149] – Relationship editor AR popup: only previously used *artists* are remembered
  • [MBS-7303] – Missing recordings in release (after updating the release)
  • [MBS-7405] – Inline search claims there are no results when the search returns an error
  • [MBS-7578] – Regression: Work types list is no longer sorted alphabetically
  • [MBS-7605] – Release editor allows invalid characters to sneak into the database
  • [MBS-7649] – Edit alias edits don’t display properly if the alias has since been removed
  • [MBS-7651] – Alias edits show no value for various fields
  • [MBS-7652] – edit_series_fk_edit and edit_series_fk_series are missing
  • [MBS-7653] – MB.utility.joinList should use the same logic/strings as the comma_list MACRO
  • [MBS-7665] – annotation edit changelog shows object reference ARRAY(0xa190008) instead of actual input text
  • [MBS-7666] – DBDefs.pm.sample is still at schema version 19
  • [MBS-7667] – GenerateSQLScripts doesn’t check for the existence of CreateTables.sql
  • [MBS-7669] – Whitespace not trimmed for series number in the relationship editor

Improvement

  • [MBS-1943] – Reorganize the sidebar to show the most important information first
  • [MBS-2474] – Display artist legal/performance name more prominently
  • [MBS-2602] – Show recent labels/artists in lookup popups
  • [MBS-6277] – Move release group relationships section for releases to the overview tab
  • [MBS-6731] – Add link to doc/Release/Format from Tracklist tab
  • [MBS-7668] – Add a stat for number of MBIDs known.

Server update, 2014-06-09

A day late in actually releasing it, but we have a new server release! Assorted bug fixes, improvements, and refactoring make up this release. Thanks to the MetaBrainz team and supplementary work from chirlu, reosarevok, and nikki. The git tag for this release is v-2014-06-09. The full list of fixed tickets:

Bug

  • [MBS-4669] – Webservice returns an ASIN that was added in a voted down edit
  • [MBS-5526] – Diff highlighting messes up descriptions in edit relationship type edits
  • [MBS-7455] – ISE for /ws/js/edit/preview: Error: Could not create MusicBrainz::Server::Edit::Release::DeleteReleaseLabel edit
  • [MBS-7552] – Entering a Series that requires disambig. changes the part of-numbering
  • [MBS-7574] – Regression. Can’t fill release group field by pasting URL
  • [MBS-7583] – Nonfunctional relationship page for instruments
  • [MBS-7584] – Double “Relationships” heading on label relationships page
  • [MBS-7585] – Missing series in subscriptions page
  • [MBS-7588] – ARRAY(0x1987d1c8) shown on add work edit
  • [MBS-7589] – Edit types in edit search are sorted in ASCII order
  • [MBS-7590] – Search results for instruments should strip markup from the description
  • [MBS-7604] – Whitespace+invalid character at the end of a string causes an ISE
  • [MBS-7608] – Internal server error looking up CD TOC with too large offsets
  • [MBS-7609] – “Refine this search” links breaks for entities with no entity filter in the edit search
  • [MBS-7611] – Internal server error loading annotation revision
  • [MBS-7614] – Relationship attributes are validated against the old link type
  • [MBS-7615] – Changes to URLs only are treated as no change
  • [MBS-7624] – Possible to create edits which add two unknown country release events
  • [MBS-7633] – l(‘Series’) is ambiguous
  • [MBS-7636] – Regression + data corruption: edit page swaps the direction of relationships

Improvement

  • [MBS-1377] – Turn relationships with multiple instrument/vocal attributes into multiple relationships
  • [MBS-7541] – Add Series to the statistics
  • [MBS-7542] – Add instruments to statistics
  • [MBS-7595] – Discid edits should show release artist
  • [MBS-7618] – fullexport generation must complete before replication packets are copied, resulting in temporary but several-hour gaps in the available replication packets
  • [MBS-7621] – Treat non-free-text work attribute types with no options as grouping types
  • [MBS-7631] – Make instrument relationships link back to the instrument entity pages

New Feature

  • [MBS-4825] – Normalize blogspot URLs

Task

  • [MBS-7499] – Add vkdb.jp to the otherdbs whitelist
  • [MBS-7571] – Factor out lists of entities in the code

May 2014 virtual machines released

I’m pleased to announce that we’ve just released the May 2014 Virtual Machine images of the MusicBrainz server!

If you’d like to direct download or BitTorrent download these images, head on over to our Server Setup page.

There have been no real changes to the setup of the VM images — we still publish a VirtualBox and a VMWare image. The instructions for using the VMs haven’t changed; the latest data and the May 26th version of the software are loaded on these images.

Have fun!

Server update, 2014-05-26

We’re back with another release on the heels of our schema change. This release includes some fixes that were hot-fixed to the production servers late the other week as well as some that have gone through the usual beta process. As such, most of the changes have to do with new or changed features of that release, plus some assorted refactoring and other bugfixes. Thanks to chirlu, nikki, and the rest of the MetaBrainz team for their hard work on this release!

Happy series editing!

The git tag for this release is v-2014-05-26.

The full list of fixed tickets, as usual:

Bug

  • [MBS-6181] – Deleted editors may be nominated for auto-editorship
  • [MBS-6309] – Search results don’t have a score column anymore after a few pages
  • [MBS-7354] – Internal Server Error in mini relationship editor when a relationship has been removed
  • [MBS-7513] – Uploading PDF cover art requires the user to switch to the “All files” filetype in the file selection dialog
  • [MBS-7514] – New accounts may be called “Deleted Editor #nnn”
  • [MBS-7516] – Age of 0 not displayed in user profiles
  • [MBS-7519] – “PDF file” labels displayed on top of image previews
  • [MBS-7530] – Caught exception in MusicBrainz::Server::Controller::Series->add_alias “The edit could not be created.
  • [MBS-7532] – ISE: Undefined value when relating Release Groups to an ordered Series
  • [MBS-7533] – Can’t create a new series from inline search
  • [MBS-7538] – Regression: Not possible to edit URL relationship attributes
  • [MBS-7539] – series. numbering elements is not auto-edit while setting automatic is
  • [MBS-7543] – “Cancel” does not cancel changes made in mini-relationship-editor for instrument-performer ARs
  • [MBS-7544] – Grouping link types are selectable in the release editor
  • [MBS-7545] – Release editor creates empty URL edits if you just change the release name
  • [MBS-7548] – Automatic series ordering doesn’t work correctly
  • [MBS-7549] – Regression: Release Editor URL seeding does not work anymore
  • [MBS-7550] – Regression: Editor search is broken
  • [MBS-7555] – ISE when changing series order numbering
  • [MBS-7556] – URL cleanup broken on edit URL pages
  • [MBS-7558] – Relationship-editor shows “The end date cannot preceed the begin date” incorrectly
  • [MBS-7559] – Sort name guess case is missing from series alias pages
  • [MBS-7561] – Relationship editor doesn’t show the count of selected recordings
  • [MBS-7562] – Regression: Error trying to unset a label in “edit release label”
  • [MBS-7563] – Multiple failures on “edit label”
  • [MBS-7570] – Add series and instruments as linkable entities in annotations
  • [MBS-7576] – Not possible to add relationships to a non-label from a label edit page
  • [MBS-7579] – Error while trying to add a release
  • [MBS-7593] – Regression: Seeding URLs to mini-relationship editor is broken
  • [MBS-7597] – Replacing recording-work relationship in relationship editor fails
  • [MBS-7599] – Relationship editor doesn’t load relationships for some recordings
  • [MBS-7600] – Firefox: “Edit Note” does not show Add Release Label if the annotation gets removed
  • [MBS-7603] – Replication trigger missing for link_attribute_text_value table

Improvement

  • [MBS-7493] – Redirected MBID pages, slave/beta servers, and non-preferred scheme should use rel=canonical to link to the main site
  • [MBS-7517] – Deleted editors should appear as “deleted”, not as “Limited User”
  • [MBS-7567] – Provide “copy/sort” bubble for all alias sortnames
  • [MBS-7575] – Support creating/editing URL relationships with the /relationship-editor endpoint
  • [MBS-7596] – The relationship editor dialog should work on a copy of the relationship until the dialog is accepted
  • [MBS-7619] – admin/config.sh means configuration is split between files (and since checked-in complicates customization)

Task

  • [MBS-7547] – Remove MAC authentication for WS

MHD San Francisco 2014

MusicBrainz is represented at Music Hack Day San Francisco 2014 this weekend!

If you’re a hacker who is participating, this page collects a number of interesting things about MusicBrainz that lets you get started with MusicBrainz easily.

If you’d like some help with MusicBrainz, come see Rob in the the back of presentation area. Rob is the guy with the crazy hair and the bright orange t-shirt.

Have fun!

Search Server Update, 2014-05-14

Bug

  • [SEARCH-141] – Remove or change setMergeFactor option in IndexBuilder
  • [SEARCH-332] – target element for relationships included in /ws/2/url endpoint is improperly structured to match lookup
  • [SEARCH-336] – searching “Universal Music” and 34. entry is correct “Universal Music”
  • [SEARCH-339] – MB search requires minimum of 4 characters to consider second name
  • [SEARCH-347] – Countries should be sorted before cities in Area search
  • [SEARCH-348] – Full Search Index Failure

Improvement

  • [SEARCH-265] – Add editors to the indexed search
  • [SEARCH-341] – Allow searching releases with specific packaging type
  • [SEARCH-352] – Include disambiguation in work index
  • [SEARCH-355] – Label Sortname field removed from Database
  • [SEARCH-356] – Area Sortname field removed from Database

New Feature

Task

  • [SEARCH-338] – Return containing areas in area search

Server upgrade and schema change, 2014-05-14

Hello again! This fortnight, as we’ve mentioned among the past few posts, is our twice-annual schema change release! This means that we’ve got some big changes, as well as special upgrade instructions.

In the former category, we’ve added support for two new entities: Series and Instruments, each of which is exactly what it sounds like. We’ve also done some various cleanup (tables missing from replication, making some attribute-style tables (e.g., label types) trees, and more properly orderable, removing unuseful sortnames). In non-schema-change fixes, we’ve fixed some merging for Artist Credits, show Area names after Places routinely, and perhaps most excitingly, added smaller versions of the relationship editor for other entities. It’s now possible to add relationships to any entity from most entity edit pages, expanding on our previous addition of URL editing!

As far as upgrading:

  1. Ensure you’ve replicated up to the most recent replication packet available with the old schema. (if you’re not sure, run ./admin/replication/LoadReplicationChanges and see what it tells you).
  2. Take down the web server running MusicBrainz, if you’re running a web server.
  3. Turn off cron jobs if you are automatically updating the database via cron jobs.
  4. Make sure your REPLICATION_TYPE setting is RT_SLAVE
  5. Switch to the new code with git fetch origin followed by git checkout v-2014-05-14-schema-change
  6. Run ./upgrade.sh (or carton exec -Ilib -- ./upgrade.sh if you’re using carton, with older installs).
  7. Set DB_SCHEMA_SEQUENCE to 20 in lib/DBDefs.pm
  8. Turn cron jobs back on, if needed.
  9. Restart the MusicBrainz web server, if applicable. It’s also recommended you restart memcached.

The git tag for this release, as mentioned above in the instructions, is v-2014-05-14-schema-change.

Full release notes, as usual:

Bug

  • [MBS-5978] – Replication feed is missing release_tag
  • [MBS-6709] – “None” is no longer the last Packaging type after adding Book and Cassette Case
  • [MBS-7482] – Artist merge with AC renaming does not merge identical ACs

Improvement

  • [MBS-2410] – Label types not a tree anymore
  • [MBS-2714] – Add support for Series.
  • [MBS-5897] – Make it possible to see edit JSON in the case of an ISE or poor data display
  • [MBS-6144] – Remove the apparently-unused script_language table
  • [MBS-6602] – Remove sortnames from areas
  • [MBS-6603] – Remove sortnames from labels
  • [MBS-6651] – Make it possible to disable dates for relationship types
  • [MBS-6886] – Display area after place names
  • [MBS-6887] – Model coordinates without nullable latitude and longitude
  • [MBS-7205] – Link types should track assumed cardinality
  • [MBS-7411] – Don’t require disambiguation comments for places from different areas
  • [MBS-7470] – Merging/combining RG types (primary/secondary) is unintuitive

New Feature

  • [MBS-3674] – Make instruments entities
  • [MBS-6234] – Add a relationship editor to artists, labels, recordings, release-groups, places, areas and works

Task

  • [MBS-7441] – Check non-replicated changes to DB that have happened since last schema change

May 14 schema change release time

The schema change release is approaching fast and we’re making good progress taking care of all the tasks that need to be completed. During yesterday’s meeting we agreed on the time to do this release: 14 May 1600UTC (9am PDT, 11am Chicago, 17h BST, 18h CEST, timezone map).

We will start the release process at this time — the exact time when the site will be down will depend on how long it will take us to walk through our pre-release checklist. We will be announcing the exact downtime via Twitter (@musicbrainz) and via a banner on the main MusicBrainz pages.

We expect our downtime to be somewhere between 15-30 minutes.

Thanks!

Server update, 2014-04-28

Once more now before our upcoming schema change, we’re here with our fortnightly musicbrainz-server release! Some notable changes aside from bug fixes include a few additions to whitelists newly-approved, and enabling thumbnail display for PDF cover art. Thanks to the Internet Archive for making PDF thumbnails work, and thanks to chirlu, nikki, reosarevok, and bitmap for their work this release.

The git tag for this release is v-2014-04-28.

Full list of tickets:

Bug

  • [MBS-4151] – Publisher ARs for different entity types create quite confusing label AR pages
  • [MBS-6166] – Deleted accounts can still have details edited
  • [MBS-6508] – Artist/label create/edit edits fail to display removed or merged areas
  • [MBS-7037] – Chrome displays broken image instead of error message
  • [MBS-7352] – Release editor prints an untrue error “not logged in” after clicking “Enter Edits”
  • [MBS-7376] – Selecting value from a drop-down list by pressing Enter prompts inexplicable behaviour
  • [MBS-7400] – “merge artist” edit with “rename artist credit” option is a tracklist changing edit but is missing in release, release-group and recording edit histories
  • [MBS-7412] – Reorder medium edits are generated for new releases
  • [MBS-7437] – Area can’t be removed from a place
  • [MBS-7438] – Empty edits are created for places due to coordinate padding
  • [MBS-7439] – Internal server error when seeding badly formatted dates to the release editor
  • [MBS-7444] – Release editor seeding should ignore medium position field
  • [MBS-7445] – Not all mediums gets reordered when you delete one
  • [MBS-7447] – Internal server error when seeding invalid track lengths to the release editor
  • [MBS-7450] – Places on the equator don’t have their coordinates displayed
  • [MBS-7451] – Possibility to remove track durations from releases with discID
  • [MBS-7453] – Internal server error in the release editor (new row for relation “release_group” violates check constraint “control_for_whitespace”)
  • [MBS-7455] – ISE for /ws/js/edit/preview: Error: Could not create MusicBrainz::Server::Edit::Release::DeleteReleaseLabel edit
  • [MBS-7456] – Error when using the track parser to parse artists and it can’t find any
  • [MBS-7458] – Internal server error when trying to delete a non-existent user
  • [MBS-7459] – Internal server error displaying add disc ID edit (Can’t call method “release” on an undefined value)
  • [MBS-7464] – Release editor does not use canonical URLs

Improvement

  • [MBS-4758] – Show list of recordings after work rels on work pages
  • [MBS-5552] – Mark edit URL merge-edits properly
  • [MBS-5640] – Don’t embed full size images in the page when no thumbnails are present
  • [MBS-7345] – Sidebar external links: show span.mp pending edit status
  • [MBS-7361] – Provide an HTML markup hook for finding/matching dynamic work attributes
  • [MBS-7460] – Enable thumbnails for PDFs

Task

  • [MBS-7440] – Add Resident Advisor and IMVDb to the otherdbs whitelist
  • [MBS-7454] – Add Rap Genius / Rock Genius and Project Gutenberg to lyrics whitelist

Sub-task

  • [MBS-7448] – Add URL lookup to the website