Category: NGS

Ollie and I have been working on a bunch of schema changes to make things more consistent and to add more time stamps to the database. We’ve included other data migration patches from Luks and Murdos as well.

All of these changes have been pushed to the master branch on our git repository. If you pull this branch from our repo, you must reload your data with this fresh NGS snapshot!

On monday we will discuss if we can call our core schema frozen yet. Let’s hope so. 😉

We’re working hard to get to the Release Candidate 1 release for NGS. We’ve got lots of code reviews in progress, lots of code being committed and a lot of activity all working towards shipping NGS.

However, as we’re working an exposing new features we’re still getting a number of new bugs submitted that expose other issues that we need to address. This is the problem with porting a very large old codebase — we need to simply slog through it and get it done as soon as we can.

With those caveats, I’ll say that I expect the RC1 release to happen somewhere near the end of November. When we get closer and our bug list is converging, I’ll post an actual date for this release. In the meantime, we’re going to put our heads down and get more work done.

Sorry for being vague about this — but I would much rather not slip anymore releases.

I’ve just pushed out a new NGS data dump.

We’re finishing up work on the migration scripts and we’re hoping to finalize the last schema change for NGS next week. After that the NGS schema will be officially frozen in anticipation of the release. After this change I will update the data on the test server and restart the replication feed for one (hopefully) last time.

Stay tuned for more updates!

I’m sorry to say that we’re running a bit behind in bringing you MusicBrainz NGS Beta 3, but we’ve finally managed it!

I am, however, proud to say that we’ve made incredible progress in the last few weeks. Not only are Warp and Acid2 rounding out our features nicely, but with Nikki and Navap backing us up we’ve found/fixed a ton of problems and made countless improvements to the look of the new site. While at times it seems that we’re working on Duke Nukem NGS that seems to never get finished, it is days like today that make me very happy with what we are going to deliver when NGS ships. We’re making sure that the software we deliver at the end of this massive process is solid and will make our users happy. In that sense I am very happy that our users and customers are giving us the time to finish this project properly.

Enough waxing poetic — now for the meaty details of this release! Take a look at the massive change log that shows all the issues we’ve tackled since beta 2. The beta 3 release is available for your testing pleasure on our test server. If you’ve not logged into the test server before, use your normal password from the MB site — the data on test is a little bit old so if your password doesn’t work, try using one that you’ve used a couple of months ago.

This release is solid enough for people to come in and have a serious play. Please come test your favorite features and if you find bugs, please report them to Jira, our bug tracking system. (click on Create Issue in the upper right hand corner). We’ve fixed a ton of bugs already and we’re not afraid to have you report new ones! Beta 3 adds more features and improves many other features, but it is still not feature complete. Our next release, Release Candidate 1 will be feature complete — I hope to set timeframe for this release within a couple of weeks.

And now for some quick notes on areas of NGS that we’ve been focusing on:

• Edit system: The edit system is now looking stable and fairly complete, though it is still lacking batch operations, which were hoping to get in for beta 3. There is also support for migrating all pre-NGS edit types, though some have display issues. (we’ll be loading new data onto test soon and then we’ll re-run the edit migration script).
• Web services: The backwards compatible web service (/ws/1) is feature complete up to cd stubs, but everything else should just work (please point tagger applications at test and see what happens!). The new web service (/ws/2) is feature complete except submitting CD Stubs, ISRCs and collections — please take a look at the specification for Web Service Version 2 and see if you can break it!
• Release editor: The release editor in beta 3 has two major new features. First is the a recordings tab, which allows you to associate recordings with tracks. Second is a preview of your edits on the final tab.
• CD Stubs: Browsing top CD Stubs and viewing CD Stubs is now working. Submitting/importing CD stubs is not complete yet.
• Lists (formerly collections): Collections have become a more generic concept of Lists, which allows multiple lists of releases per user. (This was one of our GSoc projects).
• RDFa: Artist and release pages now contain embedded RDFa! For more details, please see our wiki page for the RDFa feature.

Thank you to everyone who has worked on this release! Things have been a little stressful the past few days as we tried to push as many things out the door as possible, but the results have been quite worthwhile. Now it is time to relax a little before we start charging towards our Release Candidate 1 release. Thanks everyone!

UPDATE: I added a note about the RDFa support.

After our weekly meeting we’ve decided that we’re going to release Beta 3 this wednesday. Stay tuned for more info on Wednesday!

What would a deadline be without a delay? Sadly, we felt that we were not quite ready to unleash beta 3 today. Rather than unleashing software that isn’t ready, we’re opting to delay beta 3.

We’re taking a look at where we stand and we hope to be able to finish it for next week. I’ll post about an update hopefully tomorrow.

After a 3 hour bug triage session with the team we’ve managed to sort out the bugs and come up with a schedule for the next release. After looking at the number of bugs for us to tackle before the release, we felt that we really needed to have one more beta release before we are ready to call it a release candidate. Given that, we’re going to do yet one more more beta release on 7 September.

I know, another beta release? Yes, we’re sorry that this is stretching out longer than it should have, but we’re dedicated to releasing the software when its ready and not before. We’ve done our best to cut out unnecessary features, but we still have about 8 weeks of work to do. Fortunately, both Oliver and Warp are now working full time and and fully focused on this release. Re-writing a 10 year old codebase isn’t an easy thing to do and so far it looks like NGS will become stable before perl6 does. 🙂

The release candidate 1 will be 100% feature complete with no new features or tweaks being added. All we will do to the release candidate is fix bugs! For this reason we are not creating an NGS release milestone in Jira. That way, no one can postpone any tickets into the final release. 🙂

To see what we have planned for the upcoming releases, please take a look at the bug list for beta 3 and the bug list for release candidate 1. Currently we expect release candidate 1 (and hopefully only 1) about 4 weeks after beta 3. We hope to then release NGS onto the servers 2-4 weeks after that.

Sadly, the last round of NGS testing caused some problems. I’ve fixed those problems and restarted the replication feed again. Follow these instructions to get your replication started again:

• Download and install the mb_server source code from git. Follow these instructions. (Make sure you update this code from the code you may have pulled yesterday)
• Set your server type in DBDefs.pm to RT_SLAVE
• Download and import this dataset. This is the same dataset has in the last NGS test!
• Insert this required row into the database, using our psql program:

  cd <mb_server_src>/admin
  ./psql READWRITE
  insert into replication_control values (1, 12, 100, now());
  

• Now run admin/replication/LoadReplicationChanges a few minutes after the hour to keep up to date with the data on the test server. Please note that this system may not be stable yet and that we will occasionally load new data on our test server, which will require you to reload the data on your server.

Good luck!

Now that we’re (hopefully) done making schema changes to NGS, its time to give the replication testing another try. If you’re interested in testing replication for NGS, follow these instructions:

• Download and install the mb_server source code from git. Follow these instructions.
• Set your server type in DBDefs.pm to RT_SLAVE
• Download and import this dataset.
• Insert this required row into the database, using our psql program:

  cd <mb_server_src>/admin
  ./psql READWRITE
  insert into replication_control values (1, 12, 0, '2010-06-15 17:00:02.795317-07');
  

• Now run admin/replication/LoadReplicationChanges a few minutes after the hour to keep up to date with the data on the test server. Please note that this system may not be stable yet and that we will occaisionally load new data on our test server, which will require you to reload the data on your server.

Good luck!

I’m pleased to announce that we’ve finished hacking on NGS Beta 2! The test server is updated with the beta 2 source code and a fresh data dump (20100616 complete with 119,000 old edits migrated).

To play with NGS Beta 2, head over to the test server and log in with your normal MusicBrainz username/password. If you find any probems, please enter a bug report in our bug tracker. To see the complete list of issues that have been resolved for Beta 2, see this rather long list.

Oliver says:

Here’s a brief summary of my changes:

• Edit migration – old historic edits are now migrated, along with votes. Edit notes are not yet migrated, though work has begun in this area
• Many bug fixes from beta 1, including improved error handling
• A few more possible edits, including removing PUIDs and adding/deleting ISRCs from recordings
• Case change edits are now auto-edits
• Aliases now have a ‘locale’ field
• Annotations can be previewed before submit

It’s hard to summarize the apparent 160 issues in JIRA though, but these stand out at least.

Warp says:

Completely new release editor, not quite finished but it should give you a good feel of how the final version will work.

Some big changes to /ws/2, see the new specification.

Non-latin names are now properly sorted.

Green tagger icons

Ratings can be cleared.

Lots of data display fixes and tweaks.

Both Oliver and Warp have been working hard to meet this deadline, and I’m pleased that they’ve completed this important milestone! Many thanks to Oliver, Warp, Nikki, Navap, Ijabz, Murdos and anyone else who had a hand in working on beta 2. I would also like to thank Alisa Lemberg specifically for her efforts in helping Warp, Navap and myself redesign the release editor. Alisa’s UX experience helped us focus our efforts and finally deliver a solid release editor. Without her we’d still be guessing the right approach to take for creating a usable release editor.

Happy Testing!