From BCCD 3.0

Jump to: navigation, search

This page is to document the processes behind being the release engineer for the BCCD. Our plan is to cycle the position every few releases so that every developer is familiar with what's required.




Ideally there would be a single process regardless of release type. However, in practice there are two (high-level) types of releases: Major and Minor. Fortunately only the first few steps differ.

In the examples below, we'll be operating with the following assumptions

Create release candidate (Minor)

Since the entire point of having release versions -- especially minor versions -- is to quickly and easily determine what changed between two consecutive versions, we want to make sure that the release candidate we create has a well defined set of changes.

  1. Identify the changes necessary -- i.e., compile a list of Subversion revisions that match the changes you want for your new release
    • Typically you'll have a milestone created for a specific release (e.g.,, and with that milestone will be associated a set of tickets. If developers have been following the commit rules, then all of these tickets should have, in the comments, a list of the subversion revisions relating to that ticket. You can use this script to fetch all revisions associated with a list of (space separated) ticket numbers.
    • Some developers might not follow the commit rules of adding a ticket number into their commit messages, so you should also check the timeline for changes that you need.
  2. Copy the previous version in the repo. This operation happens entirely server-side, but should be done on hopper anyway:
    svn copy ^/bccd-ng/tags/bccd-3.0.{3,4-rc}
    • Note: There's a -rc on the end of this new path. Until it's tested, this will just be a Release Candidate.
  3. Add a new apt repository, and copy the old one as appropriate
  4. Checkout the new version. We won't be able to merge the changes without a working copy:
    svn co ^/tags/bccd-3.0.4-rc
    • You should do this in an appropriate working path, e.g., ~/svn-cluster/bccd/
  5. Merge in the new changes.
    • There's a script available here that can help, but as with most automated systems, under most circumstances, it doesn't know as much as you do and will mess up. Contact fitz if you want help with using it.
    • Often better is to do it manually:
      cd bccd-3.0.4-rc; svn merge -c <comma separated list of revision numbers> ^/bccd-ng/trunk .
      • This assumes all of you revisions are in trunk (which they often aren't), so substitute that where necessary (one thing the script does is figure this out for you).
      • Make sure to list the revision numbers in ascending order or you will risk having to do some extra merging.
    • Or if the revisions span the entirety of a range:
      cd bccd-3.0.4-rc; svn merge -r[rev1]:[rev2] ^/bccd-ng/trunk .
      • Where [rev1] is ONE LESS THAN the lowest change in the range and [rev2] is the highest.
    • Sometimes there are few enough changes that you can copy in the new files manually (for example in a point release that fixes something in the bccd user's .bash_profile) and you can just do an svn cp instead of a merge (simpler, easier, and less error prone!!)
  6. Fix all of the conflicts that the svn merge creates. For tree conflicts, use svn resolve --accept working
  7. Edit the CHANGELOG (root of the branch) to reflect the new changes you've just added it.
  8. Commit the changes:
    svn ci -m "merge in changes for milestone:3.0.4"
    • Note: Trac will automatically turn "milestone:3.0.4" into a link.

Now you have a release candidate branch! We'll cover creating the ISO for testing later.

Create release candidate (Major)

Creating a release candidate for a major version (e.g., 3.1) will be more complex than for a minor version as, by definition, it has far more going on. That said, you could still follow the steps above for creating a Minor RC, but it's likely to be a lot more work. Some of the steps here fairly high level, but duplicate information above; see up there for specifics.

  1. svn copy trunk to a new tag with your version number. The reason we're copying trunk and not the previous version as above is because in most cases, the reason we're making a new Major version in the first place is because significant changes have happened. Doing a significant number of merges is highly error prone and not recommended.
  2. Verify -- either by talking to Bccd-dev email.png or by checking the timeline -- that there aren't changes you need outside of trunk.
  3. Merge in those changes (if applicable) to your new RC branch. Requires a checkout of said RC branch.
  4. Update the CHANGELOG
  5. Commit those changes (if applicable).

Now you have a new Major Release Candidate!

Generate an RC ISO

As of May 2011 there exists a mechanism for sending a properly formated email to and get an ISO generated for you. See this script for excruciating detail. If you're a member of the software group on hopper (i.e., you have permissions to commit to the repository) you can send an email to this address with a subject line of "build <branch>" (without the quotes) and that branch will be built for you. See the comments here for what format the branch name is expected in. For our purposes here, "tags/bccd-3.0.4-rc" is appropriate.

If you don't want to rely on the script (or it's not working), here's the manual process for generating an iso:

  1. Login to Jenkins on bigfe
  2. Copy a previous build
  3. Go to the Configure page for the new build
    • Update the SVN directory
  4. Update bin/build_livecd.conf:
    • SUITE should be changed if the Debian target for the release has changed from the last release
    • OUTDIR should be set to /cluster/bccd-ng/testing/$(whoami)
    • WEBSVN should be the URL of the release tag
    • RELEASE should be end with a -rc suffix

Now you should have two (one for each architecture) ISOs ready for testing! Grab the URLs from and send them out to the list for testing!

NOTE:- After the testing if any of the bugs get fixed. Then you have take the following steps to generate another set of fixed ISOs:

1. cd /path/to/revision-branch
2. svn update
3. svn merge -c <revi1, revi2,...> (comma separated the revisions that were made to fix the a bug).
4. Make Sure to commit your changes
5. run a fresh build following instructions above.


Once testing is done and approved, you're ready to push it into production.

  1. Update bin/build_livecd.conf no longer has a -rc suffix
  2. Rename the tag directory so it no longer has a -rc suffix: svn rename bccd-3.0.4{-rc,}
  3. Update the repository URL in both the amd64 and i386 Jenkins builds to point at the new location
  4. Change directories to the stable ISO directory:
    $ cd /cluster/bccd-ng/stable
  5. Rename the ISOs and their MD5 sums such that "-rc" is not in the title:
    $ mv $RELEASENUM-rc.amd64.iso $RELEASENUM.amd64.iso
    $ mv $RELEASENUM-rc.amd64.iso.md5 $RELEASENUM.amd64.iso.md5
    $ mv $RELEASENUM-rc.i386.iso $RELEASENUM.i386.iso
    $ mv $RELEASENUM-rc.i386.iso.md5 $RELEASENUM.i386.iso.md5
  6. Remove the oldest ISOs in /cluster/bccd-ng/stable so that only the two most recent versions are there (to save disk!!)
  7. Update the CHANGELOG in that directory.
  8. Update the website:
    1. Browse to
    2. Log in (should just be your hopper username/password, if it doesn't work, contact Bccd-dev email.png)
    3. Edit the HTML for that page to point to the new version. Make sure you also update the date, size of the ISO, and the link to the MD5.
    4. Save.
  9. Send an email to announce the new version!

Misc Notes

Tickets and milestones

Each release is tied to a milestone in Trac and each milestone has some number of tickets associated with it. It is the release engineer's duty to manage these associations and to make sure each release has reasonable fixes and improvements.

The ability to create and associate tickets with milestones is dependent on the MILESTONE_ADMIN action group in Trac. The release-engineer permission group has access to this (along with other goodies). When you become release-engineer, you (or a ccg sysadmin) should run

  trac-admin /cluster/trac/bccd-ng/ permission add $(whoami) release-engineer

When you pass on the release engineer hat, you should run

  trac-admin /cluster/trac/bccd-ng/ permission remove $(whoami) release-engineer

Before associating tickets to a milestone, you must first create one. Got to the roadmap page in Trac, and click on "Add new milestone." Give the new milestone a name based on the version number for a release (e.g., 3.0.2). See the prose posted here for what the version numbers mean.

Within the "Modify Ticket" section of a ticket's page there is a drop-down menu for a Milestone. Choose reasonable tickets for each milestone. What "reasonable" means will vary per release and the tasks that need to be done.


Check out the release script to see what kinds of specifics are necessary. The script takes care of most details -- copying ISOs, creating the tag, etc.

What it does not handle is:


A forwarding alias release is setup on If possible, use this as the Reply-To to ensure continuity through release engineer changes.

Personal tools