Schema 17/18 upgrade instructions

We’ve just completed our extra schema upgrade. The full instructions for upgrade follow:

Schema 16 to schema 17 upgrade

If you already ran the migration that was announced May 15th, or if you imported a data dump from May 15th or later, skip to the next section.

  1. Run replication with carton exec -Ilib -- ./admin/replication/LoadReplicationChanges until it cannot apply any packets in schema 16.
  2. Take down the web server running MusicBrainz, if you’re running a web server.
  3. Turn off cron jobs if you are automatically updating the database via cron jobs.
  4. Make sure your REPLICATION_TYPE setting is RT_SLAVE in lib/DBDefs.pm
  5. Switch to the new code with git fetch origin followed by git checkout schema-16-to-17
  6. Run carton install --deployment to install any new perl modules.
  7. Run carton exec -Ilib -- ./upgrade.sh from the top of the source directory.
  8. Set DB_SCHEMA_SEQUENCE to 17 in lib/DBDefs.pm
  9. Turn cron jobs back on, if needed.
  10. Restart the MusicBrainz web server, if needed.

Schema 17 to schema 18 upgrade

  1. Run replication with carton exec -Ilib -- ./admin/replication/LoadReplicationChanges until it cannot apply any packets in schema 17.
  2. Take down the web server running MusicBrainz, if you’re running a web server.
  3. Turn off cron jobs if you are automatically updating the database via cron jobs.
  4. Make sure your REPLICATION_TYPE setting is RT_SLAVE in lib/DBDefs.pm
  5. Switch to the new code with git fetch origin followed by git checkout v-2013-05-24
  6. Run carton exec -Ilib -- ./upgrade.sh from the top of the source directory.
  7. Set DB_SCHEMA_SEQUENCE to 18 in lib/DBDefs.pm
  8. Turn cron jobs back on, if needed.
  9. Restart the MusicBrainz web server, if needed. EDIT: also restart memcached here, see http://tickets.musicbrainz.org/browse/MBS-6376

Note that the tags to check out for the two migrations are different.

Changes

For the list of changes in schema 17, see the former blog post. The changes for schema 18 are:

  1. Fix the track table corruption that the schema 16-17 upgrade created, by importing a copy of the ‘track’ table from the production database.
  2. Fix some indexes and constraints that should not be on slaves or which had bad names starting with ‘medium2013’ or ‘track2013’
  3. Create a missing index on medium.release that dramatically improves performance.
  4. Fix the ref_count column of the artist_credit table, which was not updated properly at the schema 16-17 upgrade.

12 thoughts on “Schema 17/18 upgrade instructions”

  1. Note on this: doing some migration on a server for the second half (17-18), I got a “this database doesn’t correspond to any replication sequence” error when trying to replicate after the migration. If this is the case for you, you should run carton exec -Ilib — ./admin/psql READWRITE and then issue UPDATE replication_control SET current_replication_sequence = 68365.

    We’ll be looking into fixing this more correctly.

  2. Hi there,

    Just wondering, are you guys any closer to getting a new VM image created?

    Thank-you 🙂

  3. Closer, as in “the urgent firefighting is done and we can now start thinking about stuff like that”. The guy who does VMs is traveling next week and I’m unsure if he’ll have time, but hopefully! (I know we said this week, that was before we found the schema change broke stuff 😦 )

  4. @ianmcorvidae: Thanks, I had the same error and your instructions solved the problem!

  5. @ianmcorvidae: THANKS!

    Without your
    UPDATE replication_control SET current_replication_sequence = 68365
    I would wait for new replication data forever…

    For linux beginners like me:
    You have to copy&paste the above command AND then press ENTER. On the new command prompt type g and press ENTER to execute the above command. You get something like “UPDATE 1” as response. Then type q to quit the psql-prompt.

  6. It’s probably just me being a total novice, but I can’t get any of the upgrade commands to work.
    When I enter the ‘git fetch origin’ command I get a ‘fatal: Not a git repository (or any of the parent directories): .git’ response.
    So I’m stuck.
    As there is a new vm image in the pipeline I’ll be patient and wait for that

  8. reosarevok: I wasn’t, but now I am. The ‘git fetch origin’ works OK but ‘git checkout schema-16-to-17’ generates an error ‘Your local changes to the following files would be overwritten by checkout: carton.lock. Please, commit your changes or stash them before you switch branches. ‘carton install –deployment’ appears to update files and ‘exec -Ilib — ./upgrade.sh. generates ‘Error: Schema sequence must be 15 when you run this script’

  9. First of all: when you get an error, you should probably stop, generally 🙂 the errors after the first are because of the first.

    Probably you’ve done some carton stuff separately. So, first do ‘git checkout HEAD carton.lock’, and then ‘git checkout schema-16-to-17’ and continue forward from there.

    A new VM image should happen sometime next week, in any case, if you’d prefer to wait.

  10. Thanks for the suggestion and I tried that with no great success. Everything seemed to work but the updates are still not working.
    As there’s a new vm due shortly, I can wait for that.

  11. man screw this noise. I struggled for two days to put up a mirror. first the replications took over 24hrs and only got to about December of 2012. then I tried to do it manually ran out of space, since the VM only had a 2GB boot disk. after increasing the size of both disks, and using gparted to increase the size, i got back to trying to download. then i find out there are schema updates that just don’t work.

    hopefully the VM is updated. now i know why i prefer the ms world over the linux world.

  12. So I have FINALLY figured out how to upgrade the VM to schema 18 and then apply all of the replication changes from there on forward. Huzzah!

Comments are closed.