• HOWTO Release a New Version

last modified February 26, 2009 by slinkp

This is about how to release a new version of the Opencore python package.

It is not about deployment. For that, see openplans deployment notes .

VERSION NUMBERS

See http://www.openplans.org/projects/opencore/blog/2008/07/10/opencore-version-number-plan/

FEATURE RELEASE CHECKLIST

As early as possible: poll the dev list re. work in progress and put together a rough plan of what is likely to be done in time for the next release.  Ideally this should start immediately after a release.

Throughout, I assume this is a minor feature release and I refer to the release number as X.Y.0.  A major release would be numbered like X.0.0.

When you're ready to start preparing the release:

Trunk Bookkeeping

  1. ON THE TRUNK: Look over the svn log and be sure that every important recent change is mentioned in CHANGES.txt. (This often requires looking at changesets to figure out what a commit message really meant.)
  2. ON THE TRUNK: Look over CHANGES.txt while imagining that you are a sysadmin or integrator installing opencore and you don't know anything about its internals. Do the changelog entries make sense to you in that context? They don't need to be totally self-explanatory; as long as you can glean eg. "A render bug was fixed on wiki edit views", that's enough.
  3. ON THE TRUNK: Look over CHANGES.txt. If anything requires a data migration, be sure that:
    1. there's a script named appropriately like to_X_Y.py in opencore/upgrades/
    2. there's a GenericSetup upgrade step registered for this version in opencore/upgrades/configure.zcml, with the handler pointing to the above script
    3. your release version number is updated in opencore/configuration/profiles/default/metadata.xml.
    4. the need to run a GenericSetup upgrade is documented in CHANGES.txt.
  4. ON THE TRUNK: Be sure to document in CHANGES.txt if any change is backward-incompatible.
  5. ON THE TRUNK:  Run the i18ndude commands to be sure the translation files are up to date. See i18n usage in opencore .

Create the Branch

  1. Cut a release branch from the trunk, with the appropriate minor number (and no patch number), eg. branches/X.Y

    If necessary due to ongoing work on the trunk, you can cut this branch from an older trunk revision. (This may require some further work on both trunk and branch of CHANGES.txt to be sure the relevant changes are noted in the right place in these files.)
  2. ON THE TRUNK: Add a new section at the top of trunk's CHANGES.txt, marked X.(Y+1).0dev (UNRELEASED), so its minor number is one higher than the branch you just created.  (Don't literally put a +1 in there.)

    This section may be empty for now.
  3. ON THE TRUNK: Edit setup.py and set the version number to X.Y+1.0.dev
  4. ON THE BRANCH: Edit setup.py on the branch and set the version number to X.Y.0.dev where the version matches the branch you just created.

Freeze Dependencies

  1.  In a known good build of the TRUNK, use pip freeze to create a frozen set of requirements.  Pip can be used like: 
    cd opencore
source bin/activate
easy_install pip
pip freeze | sed -e "s/svn+http/http/g" > frozen.txt
  2. Keep that frozen.txt handy for the next steps. Don't commit it to the requirements trunk, those should typically be left un-frozen.
  3. Where frozen.txt points to a svn trunk or branch revision,  consider making new tags and/or releases of the relevant packages... but CHECK PYPI FIRST: pip freeze sometimes uses svn urls for things that we've already released to pypi, I haven't looked into why.  One example is topp.utils.  After making any tags or releases, update frozen.txt to use them.
  4. Create a branch of fassembler requirements for opencore-minimal. E.g. for the 0.1234 branch of opencore, you'd create https://svn.openplans.org/svn/build/requirements/opencore-minimal/branches/opencore-0.1234. This should probably be initially copied from the requirements for the previous stable release.
  5. Merge your frozen.txt into your newly branched requirements/opencore-req.txt. 
  6. Update your new requirements/opencore-req.txt to use your opencore branch instead of opencore trunk. 

Build the Branch

  1. Use fassembler to build from your new requirements set from scratch. If the build breaks, fix as needed.
  2. Do final testing on this new build.
    1. Run all unit tests (opencore and Products.listen)
    2. Do some basic browser sanity checking
    3. Run flunc tests (XXX skip this, they don't really work without full stack)

Prepare for PyPi

­Not relevant yet because easy_install opencore doesn't work properly.

­ONCE EASY-INSTALLING OPENCORE WORKS:
   * Do python setup.py develop --uninstall; python setup.py install to test whether your package works correctly as an egg.

   * Run all unit and flunc tests.

   * Fix anything wrong with the egg on your branch.

Full Stack

NOTE: I am not sure this is really the release manager's job; it's stepping into openplans deployment manager territory.  OTOH, with our current set of flunc tests, it's not possible to run the flunc tests unless you build a more-or-less full stack. And I don't consider an opencore release fully baked unless the flunc tests that ship with it actually run and pass. Long term we really need to figure out the relationship between opencore and the flunc tests!

  1. Create a requirements branch at https://svn.openplans.org/svn/build/requirements/openplans/branches

    This should contain full stack requirements based on https://svn.openplans.org/svn/build/requirements/openplans/trunk but merge in your frozen requirements in opencore-req.txt.   Make sure to leave in all the full-stack extra stuff (oc-tt, oc-cab and the like).
  2. Build a full stack from this req branch.
  3. Run all unit tests (opencore and Products.listen)
  4. Run all flunc tests
  5. When everything works, install pip and run pip freeze; use the output to check and freeze the versions of the stuff that you haven't already frozen (oc-cab and so forth).
  6. Repeat steps 2-4.  Hooray.

Check Trunk Again

What about changes on trunk since you started working? When in doubt, leave it out.

If *important* bugfixes have landed on trunk since you started doing all this, merge those to the branch and test some more.  You should insist that all such changes have tests.  When in doubt, Leave It Out.

Be very very conservative with this. I have wasted a lot of time investigating problems with trunk changes that didn't really warrant merging to the branch in the first place; and I have merged last-minute changes that turned out to be buggy.

When in doubt, LEAVE IT OUT.

If you had to fix anything on your branch, merge that to the trunk and note it in trunk's CHANGES.txt.

Final Release Prep

  1. Put today's date in the version number at the top of the branch's CHANGES.txt and remove the .dev from the version.
  2. Update the branch's setup.py removing the .dev from the version.
  3. Tag the opencore branch, eg. opencore/tags/X.Y.0
  4. ON BRANCH: Update setup.py to the *next* version +dev, eg. X.Y.ZZdev  where ZZ=Z+1.
  5. Merge your CHANGES.txt updates to the trunk.

Final Tag Test

  1. Create a tag from your opencore-minimal requirements branch but point it to your opencore tag, not opencore branch
  2. Build and test from this new requirements tag
  3. Create a tag from your openplans requirements branch but point it to your opencore tag, not opencore branch
  4. Build and test from this new full-stack requirements tag

PyPI Release

ONCE EASY_INSTALL WORKS:

  1. Create an egg from the tag
  2. Run flunc and unit tests again against your final egg 
  3. Upload the egg to pypi

Post-Release

  1. Publish a blog post at http://www.openplans.org/projects/opencore/blog/ with a summary of what's in CHANGES.txt from this release.
    Include links to the requirements profiles.
  2. Make an announcement on opencore-dev and opencore-users lists linking to the blog post.
  3. ON BRANCH: Add a new section at the top of the branch's CHANGES.txt, marked X.Y.Z.dev (UNRELEASED), where Z is 1 more than the release we just made.
  4. Go on vacation and ignore all support requests.  You surely did everything perfectly, right? 

BUGFIX RELEASE CHECKLIST

This will be a micro release, so the release number will be X.Y.Z where Z is greater than 0.
Compared to making a feature release, some steps will be skipped (marked N/A below); some will be different.

Trunk Bookkeeping

N/A

Create the Branch

N/A

Freeze Dependencies

This should have been done for the previous stable tag release.  At most you might have to update a requirement to get an upstream bugfix.  But if the release manager forgot to do this, see instructions for feature release above and do it now.

Build the Branch

Same as for feature release.

Full Stack

Skip creating the full stack requirements branch as it should already exist. Otherwise, same as for feature release.

Check Trunk Again

There must be some bugfixes on this branch. Be sure those are also merged to trunk and note them in trunk's CHANGES.txt (in the section for the unreleased next feature version.)

Otherwise same as for feature release.

Final Release Prep

Same as for feature release, except that the last point of the tag will be > 0.

Final Tag Test

Same as for feature release.

PyPI Release

Same as for feature release.

Post-Release

Same as for feature release.

­ ­