Author: acidcycles

Schema change release, 2011-07-11

Today we released our first schema change update since NGS. This change is quite a radical one, as it merges both of our databases (“READWRITE” and “RAWDATA”) into a single database. For most users of the database, this probably won’t affect you, but you’re encouraged to run the upgrade process anyway. Here’s what you need to do:

  1. Take down the web server running MusicBrainz, if you’re running a web server.
  2. Turn off cron jobs if you are automatically updating the database via cron jobs.
  3. Set DB_SCHEMA_SEQUENCE to 13 in lib/DBDefs.pm
  4. Make sure your REPLICATION_TYPE setting is RT_SLAVE
  5. Switch to the new code with git fetch origin followed by git checkout v-20110711-schema-change
  6. Run ./upgrade.sh from the top of the source directory.
  7. Install the perl modules Algorithm::Merge and Algorithm::Diff
  8. Turn cron jobs back on, if needed.
  9. Restart the MusicBrainz web server, if needed.

This process may take a while, as it has to dump one database into another, and download a few extra changes to ensure slaves aren’t missing any data. The RAWDATA database should no longer be in use and you should be able to drop it, but waiting to see that everything is working well might be a good idea.

This schema change does not introduce any new data. For everyone else, here’s a list of what got fixed since the last release!

Bug

  • [MBS-1977] – ModBot is unable to close some edits
  • [MBS-1979] – Unable to edit a “later translated versions” relationship with change direction
  • [MBS-2026] – Subscribed artists open edits won’t load for editors with large amounts of subscriptions
  • [MBS-2442] – MB postgres unaccent extension overwrites unaccent.so library shipped with postgres-contrib
  • [MBS-2689] – Timeline isn’t working in Opera 10
  • [MBS-2698] – Some tables are not replicated through Live data feed
  • [MBS-2812] – Adding track times has caused a failed dependency for an edit which changed the track titles.
  • [MBS-2826] – Web service returns malformed XML (not escaped properly)
  • [MBS-2831] – “Edit medium” edit display: Artist credits changes are incorrect
  • [MBS-2974] – I don’t receive e-mail when someone votes no to my edit any more
  • [MBS-2981] – Changing the case of recording comments should be auto-edits
  • [MBS-2986] – tracklist_index was not populated during NGS
  • [MBS-2995] – Caught exception in MusicBrainz::Server::Controller::WS::2::ReleaseGroup->release_group_browse “Can’t call method “format”

Improvement

  • [MBS-1500] – Remove tracklist_index.tracks
  • [MBS-1707] – Advanced tracklist in RE: “Title” to “Disc title”
  • [MBS-2242] – Disable editing of Medium title when there’s only one medium
  • [MBS-2434] – Combine READWRITE and RAWDATA
  • [MBS-2462] – Other edit types that don’t highlight what has changed between old and new values
  • [MBS-2583] – “Edit Medium” should show you which medium is being edited with respect to the overall release
  • [MBS-2767] – Release-group XML result should include first release date

NGS bug fixes, 2011-07-05

The wheels of change are slowly starting back up again, now that we have all our employees back 🙂 Here’s what we’ve got done since the last release:

Bug

  • [MBS-1849] – Alias locales not included in the XML
  • [MBS-2223] – Length not shown in edit when adding a standalone recording
  • [MBS-2586] – Search results do not show standalone recordings
  • [MBS-2807] – Stack trace requesting /ws/2/collection when user has no collections
  • [MBS-2846] – Internal Server Error on lot of editing histories
  • [MBS-2880] – Error viewing release group/URL
  • [MBS-2882] – Merge recording edit display incorrectly once applied
  • [MBS-2898] – Search indices aren’t updating, haven’t updated in over 20 hours

Improvement

  • [MBS-777] – Make cancelling take effect sooner
  • [MBS-1508] – Should release browse request allow inc=recordings
  • [MBS-2767] – Release-group XML result should include first release date

Beta release of libmusicbrainz4 with NGS support

Andy Hawkins and Lukáš Lalinský have been doing some work on libmusicbrainz, and a beta release of version 4 is now available. Here’s what Andy has to say:

Hi all,

In conjunction with Lukas I’ve been working on a complete rewrite of libmusicbrainz to support the new Web Service v2 released alongside NGS. I’m happy to announce the first Beta release of this new library (libmusicbrainz4).

The library is written as a relatively simple parser for the results from the Web Service, with as little logic as possible. This should mean it is easy to maintain to keep in step with any changes in the schema for the XML Web Service.

The main interface is a C++ class ‘CQuery’, that returns a Metadata object containing a parsed version of the response. There is also a C interface that loosely wraps the C++ classes.

Using the library requires a good understanding of the queries and responses to and from the Web Service.

The library is available to download here:

ftp://ftp.musicbrainz.org/pub/musicbrainz/libmusicbrainz-4.0.0beta1.tar.gz

Documentation is generated from the source (make docs), an online copy is available here:

http://users.musicbrainz.org/~luks/docs/libmusicbrainz4/

If you find any issues, please report them using JIRA (under the project ‘libmusicbrainz’) here:

http://tickets.musicbrainz.org/

I will attempt to be available via this mailing list and the #musicbrainz-devel IRC channel on freenode in order to answer questions if at all possible.

NGS bug fixes for 2011-06-20

We ended up delaying this release by a day to make sure we could cram lots of work in before Oliver goes on holiday, so we’ve got quite a lot of things fixed last week!

Bugs

  • [MBS-1463] – Show artist(s) on doing recording-recording edits
  • [MBS-1507] – Search redirects to entity on partial matches
  • [MBS-1637] – When looking up existing tracklist, use current release name/artist
  • [MBS-2009] – Ticking the Vote box of the first edit causes "Vote on all edits" to match
  • [MBS-2010] – Release editor: Add existing tracklist to release search shows duplicates
  • [MBS-2028] – Entities highlighted with no open edits
  • [MBS-2041] – Change track artists still not working as expected
  • [MBS-2111] – Recording not found in add release "Recordings" phase
  • [MBS-2143] – Adding disc by searching for existing tracklist fails to find some tracklists
  • [MBS-2168] – Summary: (no changelog) when adding Catalog #
  • [MBS-2265] – In VA releases, once the auto-track duration is triggered, unchecking ‘detect track durations’ doesn’t work.
  • [MBS-2313] – Unnecessary edit annotation edit in the release editor
  • [MBS-2360] – Release editor: found recordings list doesn’t close
  • [MBS-2424] – Dropdown Box not updating Artist Name & Depressing Enter submits request
  • [MBS-2511] – Release edit interprets numbers in track titles as times
  • [MBS-2515] – Web service doesn’t return track artist credits in release query
  • [MBS-2574] – No radio buttons for attaching CD TOC
  • [MBS-2575] – Basic editor requires a space after track numbers
  • [MBS-2576] – Some edits have links on Deleted Artist
  • [MBS-2579] – Web service doesn’t return track lengths
  • [MBS-2580] – Web service returns <length/> for recordings with no duration
  • [MBS-2628] – "guess case" works incorrectly with work titles
  • [MBS-2630] – XML web service v1 reports duplicate ISRC and PUID
  • [MBS-2649] – Not receiving subscriptions emails
  • [MBS-2728] – CDlookup proposes and allows you to attach a DiscID to a Vinyl
  • [MBS-2738] – Internal Server Error trying to view release group editing history
  • [MBS-2753] – Annotation causing an internal server error in the search results
  • [MBS-2761] – Inline search always return artist results whatever type you chose
  • [MBS-2763] – Cannot accept edit relationship edits if an end point no longer exists
  • [MBS-2771] – Crash when relating recordings where relationships already exist
  • [MBS-2784] – Fuzzy CD lookup doesn’t work
  • [MBS-2804] – "Add release label" edit not showing again
  • [MBS-2818] – No tracklist editor textbox displayed on test.musicbrainz.org

Improvement

  • [MBS-390] – Edit search is incomplete
  • [MBS-777] – Make cancelling take effect sooner
  • [MBS-1717] – Allow selecting the release group of a release in the release editor
  • [MBS-2137] – Make subscriptions emails ignore your own edits
  • [MBS-2293] – Edit display improvement: Display ISRCs and lengths on recordings merge edits
  • [MBS-2378] – Changing artists in the release editor (release and track) doesn’t _appear_ to do what users expect
  • [MBS-2460] – Timeline graph should include MusicBrainz events
  • [MBS-2621] – Guess case editor doesn’t remember options
  • [MBS-2625] – Display "Artist in MusicBrainz" as well as the artist credit in edits
  • [MBS-2643] – Merge legend and checkbox-selector in timeline graph
  • [MBS-2752] – Copyable URLs for timeline graph need to account for geometry (i.e. zoom)
  • [MBS-2757] – Fix Artist gender statistics
  • [MBS-2769] – Release Country Statistics
  • [MBS-2822] – ‘Add disc ID’ doesn’t link to CDTOC

Task

  • [MBS-951] – MusicBrainz::Server::Data::Search::external_search is taking $c. It shouldn’t.
  • [MBS-2762] – Clear artist credits from memcached

Sub-task

  • [MBS-2451] – Display artist in release group search
  • [MBS-2665] – Edits with at least one No vote
  • [MBS-2671] – Filter by edit ID / time range
  • [MBS-2675] – Filter by label

This version has a new parameter in DBDefs, so you will need to make sure your lib/DBDefs.pm has all the options in lib/DBDefs.pm.default. There are also 2 more dependencies – DateTime::Format::Natural and Data::Dumper::Concise. The tag for this release is currently v-2011-06-20-ngs-bug-fixes

NGS bug fixes for 2011-06-14

Another week, another set of bug fixes! Here’s what we’ve addressed since the last release:

Bug

  • [MBS-1231] – Age information for a person looks weird
  • [MBS-1924] – Allowed merge direction for "Merge mediums and recordings" strategy seems non-intuitive
  • [MBS-1995] – Time from TOC is calculated incorrectly
  • [MBS-2000] – CC URL relationship does not allow selection of license
  • [MBS-2049] – Docs not in the transclusion table seem to be perma-cached
  • [MBS-2062] – Menu link ‘About->Auto-editor elections’ is invalid.
  • [MBS-2169] – Label entered regardless of clearing text
  • [MBS-2261] – Release group is listed twice on an artist page
  • [MBS-2347] – Seed particular release after clicking on the recording tabs it comes back with Frequent 502 Bad Gateway
  • [MBS-2376] – Inline search results disappear when clicking on the little arrows two times
  • [MBS-2391] – You should not be able to add the same CDTOC multiple times on the same medium
  • [MBS-2399] – Problems with mediums having duplicate CD TOCs
  • [MBS-2408] – Guess Case deletes volume and part numbers
  • [MBS-2447] – "Charter Member" thing no longer shown
  • [MBS-2483] – Edit URL edit for a removed URL displays wrong
  • [MBS-2485] – no way to merge a various artist clone into the default "Various Artists" artist
  • [MBS-2529] – Orphaned release group and recordings after rejecting or cancelling a release/medium
  • [MBS-2550] – Re-add the "Subscribe to artists I add" option
  • [MBS-2552] – Allow removing a RG that has relationships
  • [MBS-2560] – Internal Server Error when removing medium from a release
  • [MBS-2644] – Internal server error while trying to add a disc ID
  • [MBS-2645] – Error in the message when trying to relate works to recordings
  • [MBS-2651] – Edit shows up in unrelated artists’ edit history
  • [MBS-2658] – Edit URL edits cannot close if the URL being edited has since been merged
  • [MBS-2659] – Edit relationship fails if the relationship has since been edited into the same relationship
  • [MBS-2705] – Adding a note to an edit which has an existing note w/o timestamp gets presented out of order
  • [MBS-2724] – IPIs don’t appear when added in Add artist edits
  • [MBS-2744] – Track removals not shown in tracklist diff

Improvement

  • [MBS-1395] – Put relationship attributes descriptions inside of an HTML "label" tag
  • [MBS-1449] – Inline Search: relate to recordings and/or works on releases
  • [MBS-1982] – Add twitter link to home page
  • [MBS-2007] – Lack of spacing between the top nav search and the dropdown to accompany it
  • [MBS-2269] – Release editor > When new disc is added, check if the last disc is empty and replace it
  • [MBS-2299] – Display one-line TOC on discId page
  • [MBS-2430] – Tagger button for standalone recordings
  • [MBS-2459] – Timeline graph needs copyable URLs
  • [MBS-2697] – Could use a link to add a VA release group
  • [MBS-2721] – Sort recordings by comment

New Feature

  • [MBS-2567] – Statistics about artist countries

Sub-task

  • [MBS-1916] – Edit release label edit doesn’t display properly

NGS bug fixes for 2011-06-01

We’ve now slowed down a bit from the hot fixes and are starting to roll out weekly releases. To kick things off, here’s what we’ve just pushed out to the main servers:

Bug

  • [MBS-1071] – Relationships with pending edits should be highlighted
  • [MBS-1937] – Subscriptions email is not sorted
  • [MBS-1960] – Historic add release group edits have an invalid data section for artist credits
  • [MBS-2046] – Global ratelimits are not applied
  • [MBS-2055] – Add release/add disc: cannot add tracks or edit track lengths; leaving the tracklist tab loses tracklists; other problems
  • [MBS-2080] – Tracks times not being shown in release editor when DiscID is present
  • [MBS-2187] – IPIs from ASCAP not accepted
  • [MBS-2239] – Newly created work does not appear on the author’s list of works
  • [MBS-2247] – ModBot shouldn’t remove artists which were recently added
  • [MBS-2305] – Release groups shown twice on tag overview page for tag with releases and release groups.
  • [MBS-2316] – Broken link in "edits for your subscription" email
  • [MBS-2339] – Editing tracklist -> error
  • [MBS-2340] – Release tags do not show up in user’s created tags
  • [MBS-2375] – Label codes on label pages don’t have leading zeros anymore
  • [MBS-2380] – file error – work/relate.tt: not found
  • [MBS-2393] – Error when searching works for certain strings.
  • [MBS-2395] – Use the link type name in /ws/2/ XML
  • [MBS-2397] – can’t watch an artist
  • [MBS-2457] – Multiple Entries of the Same DiscID, not editable
  • [MBS-2464] – "Remove recording" edit is not displaying recording

Improvement

  • [MBS-1966] – Open edits list now backwards, showing newest edits first
  • [MBS-2074] – Change the label of the button at the bottom of the page from "Vote on All Edits" to "Save Changes"
  • [MBS-2290] – Replace letter x with multiplication sign.
  • [MBS-2379] – Wording in subscriptions email is inaccurate
  • [MBS-2396] – Make it possible to return recording and work level ARs from /ws/2/release

Task

  • [MBS-2063] – Detail ws/2 ISRC submission process

Ian McEwan, one of this years Google Summer of Code students, now has his first bit of work in MusicBrainz as well:

I have created a basic graph with a subset of the core entities and some limited interactivity.

This is essentially a stripped down version of the old MusicBrainz timeline. Lots more interesting features to come with this project, and it’s showing great potential!

The Git tag for this release is v-2011-06-01-ngs-bugfixes.

NGS hotfixes for 2011-05-27

A small set of fixes to finish the week. The next release after this is currently scheduled for Wednesday. Thanks for sticking with us through the NGS launch week! Please keep the bug reports coming in, no matter how insignificant you think the issue may be, your reporting has been tremendously helpful. The Git tag for this version is ocharles/v-2011-05-27-ngs-hotfixes – the official tag will be out late Monday (I don’t have a key to sign the tag with).

One note about the Sound Juicer/RhythmBox issues – this is actually a problem with programs being compiled against libmusicbrainz2, instead of libmusicbrainz3 and there’s nothing we can do about these problems. If you are experiencing problems we recommend you compile from source, or wait/pester your distribution to update their packages.

Bug

  • [MBS-2249] – Sound Juicer no longer works with NGS
  • [MBS-2274] – Sort names being returned incorrectly for multiple artist credits on /ws/1
  • [MBS-2295] – UX: Add Missing Entities loses your selections for artists if you forget to select (or add new artist sort-name for) one of them
  • [MBS-2296] – FixTrackLength script (aka modbot automatically setting track lengths) checks are wrong
  • [MBS-2328] – When seed release editor with recording ids for tracks the Release Editor associates them with the wrong tracks unless recording ids have been allocated for every track
  • [MBS-2332] – Error trying to edit tracklist of a release to remove excesive quoting
  • [MBS-2350] – Subscribe link doesn’t work (missing id)

Improvement

  • [MBS-2353] – Update javascript code for relationships so Wikipedia URL are recognized automatically for works

NGS hot fixes for 2011-05-26

Another day, another bunch of bug fixes! I do wonder sometimes, “how could we have got so much, so wrong?” but we’re making some really good progress. Here’s what you should find fixed now:

Bug

  • [MBS-1169] – Musicbrainz NGS Server is missing a hell of a lot of HTML sanitization. (HTML/script injections galore…)
  • [MBS-1814] – Add release from disc ID adds disc ID to every disc
  • [MBS-1956] – Adding translated tracklistings is impossible
  • [MBS-1965] – You have no collections!
  • [MBS-1993] – External links not appearing on label page
  • [MBS-2023] – CD Baby not showing up in external links
  • [MBS-2027] – Amazon cover art does not display for URL added post NGS
  • [MBS-2068] – Specify that "found ### edits" in edit listing is an estimation
  • [MBS-2119] – Punctuation not shown when editing medium
  • [MBS-2209] – Tracklist preview shows empty "disc 0"
  • [MBS-2210] – Collapsed discs cause Add Disc page to appear off top of page
  • [MBS-2277] – Basic editor more likely to lose or misapply associations between tracklists and recordings than advanced editor
  • [MBS-2281] – [ws/1] release-group request change in behavior
  • [MBS-2300] – An "edit medium" edit got stuck and can’t be approved
  • [MBS-2304] – Clicking on "Artist as credited on this track" deletes the text
  • [MBS-2308] – Displaying {additional:addionally} and so on next to relationships in all relationship pages.
  • [MBS-2309] – Invalid WS request causes crash
  • [MBS-2311] – Release editor: "An artist is required on track n" when seeding parameters to release editor
  • [MBS-2316] – Broken link in "edits for your subscription" email
  • [MBS-2319] – Cannot merge multiple recordings that have a common ISRC

Improvement

  • [MBS-1978] – "External Links" display in the sidebar doesn’t work as intended

The Git tag for this version is v-2011-05-26-ngs-hotfixes.

NGS hot fixes for 2011-05-25

Thanks for sticking with us through that brief bit of downtime for editors – here’s what we got fixed today! Of interest especially is the improved support for relationship edits in artist history. Hopefully tomorrow we will clean up the subscription system a bit more, so you can get back to voting! This version is v-2011-05-25-ngs-hotfixes in Git.

Bug

  • [MBS-1945] – Relationships edits dont appear in artist history
  • [MBS-1990] – Can’t import/add certain freedb/discid releases: "An artist is required on track x" for all tracks
  • [MBS-2057] – [ws/1] track request is missing tracknumbers in release-list after ngs migration
  • [MBS-2102] – Adding a new release/tracklist that shares recordings changes existing release tracklist
  • [MBS-2129] – Edits submitted when no changes made
  • [MBS-2130] – Edit medium (to correct separate bug) didn’t remove intended tracks
  • [MBS-2135] – Send subscriptions emails at the normal time
  • [MBS-2140] – Recordings automatically changed when editing release with CD+DVD with same track names
  • [MBS-2141] – Search indices not updating every 6 hours?
  • [MBS-2177] – All recordings are forgotten if you go back to Tracklist from Recordings
  • [MBS-2198] – Using Artist Credit with no join phrase results in a server error
  • [MBS-2219] – Internal Server error trying to edit artist credit and track title in advanced error for mashup
  • [MBS-2263] – Link in a URL’s title is incorrect
  • [MBS-2270] – Internal Server Error for invalid timezone
  • [MBS-2271] – broken link on ‘Edit Relationship’ page
  • [MBS-2272] – Cannot display some edit relationship edits
  • [MBS-2285] – Internal Server Error adding/removing anything from collection

Improvement

  • [MBS-1706] – Indicate date is Y/M/D in the RE
  • [MBS-1946] – Readd old relationships phrases in database in addition to new short ones

NGS hot fixes for 2011-05-24

More bug fixes for the NGS release, keeping up with our promise of daily hot fixes! A lot of work today, but a good whack of it is still in review, so that will be coming tomorrow. Here’s what you should see on the servers:

Bug

  • [MBS-1896] – Editing Artist/Label Name capitalisation results with "Possible Duplicate Artists" warning
  • [MBS-2106] – Unable to add new release using FreeDB import: process freezes when clicking "Add disc" radio button
  • [MBS-2163] – Internal Server Error attempting to assign a title to a medium
  • [MBS-2171] – Expand Disc does not result in the correct action
  • [MBS-2179] – Remove disc in release editor tracklist pane does nothing
  • [MBS-2181] – Internal Server Error when submitting edit to remove and add disc
  • [MBS-2203] – Internal server error when importing VA release from freedb via freedb search > import
  • [MBS-2206] – Advanced editor: Can’t add tracks to new mediums
  • [MBS-2254] – Viewing the page for a tag which is only present on works gives a internal server error for template
  • [MBS-2261] – Release group is listed twice on an artist page
  • [MBS-2267] – Number of mediums doubled on ‘Disc IDs’ page
  • [MBS-2279] – Medium heading displayed twice in list of disc IDs
  • [MBS-2280] – [ws/1] release-group search returns invalid xml
  • [MBS-2284] – Disc IDs shown with duplicate CD1/CD2 headings

Improvement

  • [MBS-1717] – Allow selecting the release group of a release in the release editor

Please feel free to make use of these, if you want to do more testing, remember our test servers are also up to date! The tag for this version in Git is v-2011-05-24-ngs-hotfixes