DevelopmentInstructions

From BCCD 3.0

Jump to: navigation, search

If you were looking for information on the test suite, it's been moved here

Contents

Overview

The BCCD-NG environment is stored entirely in the Subversion database. Read-write access is provided over ssh, at

 svn+ssh://bccd-ng.cluster.earlham.edu/cluster/svnroot/bccd-ng/

Development branches are under

 svn+ssh://bccd-ng.cluster.earlham.edu/cluster/svnroot/bccd-ng/branches

with the common branch name being your-name-devel.

The BCCDv3 ISO is assembled in a single pass in the bin/build_livecd.pl script. It is composed of a series of tests from the Bccd.pm perl module, found in trees/usr/local/lib/site_perl/5.10.0/Bccd.pm. Tests take different numbers of arguments, so look at the signature of the specific test subroutine to figure that out. Keep in mind that every run_test call will have at least four arguments: the name of the test, the success return code the test should expect (which is left blank in nearly all cases for a safe default), the message the test should print out, and the first (and maybe only) argument for the test.

For historical reasons, the liberation script is at packages/liberate.pl. This probably should be moved to a more sensible location in the future. The script has the same layout as the build_livecd.pl script. Remember that the liberation should assume no network connectivity, so all files should be fetched in the ISO build. Files that are specific to the liberation can be stored in /root/liberate, which is reference by the $LIBFETCH variable.

For more information on specific tests, see the POD documentation within Bccd.pm.

Getting started

The best way to get started is to make your own SVN branch. To do that, issue a command like this:

  svn copy svn+ssh://bccd-ng.cluster.earlham.edu/cluster/svnroot/bccd-ng/trunk/ \
  svn+ssh://bccd-ng.cluster.earlham.edu/cluster/svnroot/bccd-ng/branches/your-branch-here

You can then checkout or update that branch. To checkout your newly created branch, issue a command like:

  svn co svn+ssh://bccd-ng.cluster.earlham.edu/cluster/svnroot/bccd-ng/branches/your-branch-here

Your first commit should be to change the $websvn = line in your branch's bin/build_livecd.pl from trunk to the name of your branch.

Directory structure

Recipes

  1. Add or modify a file in ~bccd
    1. All files should be in in packages/etc/skel to support multiple users.
    2. Modify or add the file in skel, and commit the change. All the files will be pulled down automatically.
  2. Change a file in /etc
    1. Add the file to packages/etc.
    2. The placement of files in /etc comes with many dependencies (when a package gets installed, whether a service gets started automatically, etc.). For this reason, you'll have to add a test to fetch the file out of WebDAV, and optionally other tests to change its ownership and permissions. There are several for loops in build_livecd.pl that you might be able to take advantage of, if your file has similar dependencies as the other files in that loop.
  3. Add a module
    1. "Master" module files are stored in trees/usr/local/Modules/3.2.6/modulefiles. You will probably never have to touch these unless you need to add another module repository.
    2. "Software" module files are stored in trees/usr/local/etc/modules, and trees/usr/local/etc/modules-remote (if you're using remote modules).
    3. Since modules are stored in a subdirectory of tree (trees/usr), all files are fetched at once and you don't have to add a separate test.

Commits etiquette

Follow two rules when you commit:

  1. Always reference a ticket number in the commit message, e.g.
    adding libnet-ip for dhcp dialog script (#358)
  2. Always test your changes before merging from your branch into trunk.

Build requirements

Note: Some default versions of the File::Temp Perl module are bugged and do not contain the newdir method. To circumvent this issue, install File::Temp using cpan.

Stages

The BCCD has a number of different configurations, called stages. Applications need to know this so they can react appropriately. The current stage is stored in /etc/bccd-stage. The current stages are these:

A proposed addition is:

Troubleshooting

bccd-report

/bin/bccd-report is a script that end users may run to gather all information that might be useful in debugging a problem. This script will send an email to bccd-developers@bccd.net with the subject "bccd-report for <host-id>". Alternatively, the end user may supply the -n option, which will generate ~bccd/bccd-report.tar.bz2. This file maybe transported to another system and emailed to the list manually. The list of gathered information is in #696.

It is designed to have as few dependencies as possible, so that even a seriously impaired system may still generate useful information. Due to this requirement, the script uses uuencode to generate an attachment. Not all mail clients understand uuencode (i.e. Thunderbird). Here is the procedure for transforming the uuencoded-data to something useful:

  1. Make sure you have uudecode installed (on Debian-based systems, this is in the sharutils package)
  2. Save the message to a file
  3. Run uudecode <file-name>|bzip2 -d -c|tar xvf -

The end user documentation is at UserInstructions#Supplying information to developers.

Regenerating KNOPPIX/KNOPPIX squashfs

Often it's desirable to generate just the squashfs filesystem, located in the BCCD ISO image at KNOPPIX/KNOPPIX. The most common reason would be to test a feature without having to wait for an entire ISO build. These are the steps necessary to do this on a Debian-type system:

  1. Make sure you have the squashfs-tools package installed.
  2. Fetch the extract_squashfs script from SVN.
  3. Extract the ISO image and containing squashfs to separate directories: sudo ./extract_squashfs -i /path/to/bccd.iso
  4. This command will print out two paths - one with "ISO" and one with "SQUASHFS". Take note of both and be sure to clean them up when you are done.
  5. Make edits in the SQUASHFS directory as needed.
    • Generate a new SQUASHFS, stick it on an existing USB stick: sudo mksquashfs /tmp/squashfs.5ETepz /mnt/usb0/KNOPPIX/KNOPPIX -comp xz
    • Generate a new SQUASHFS (see above), but instead stick it in the ISO image directory at KNOPPIX/KNOPPIX. Then re-generate the ISO image: genisoimage -pad -l -r -J -v -V BCCDv3-some-svn-rev -no-emul-boot -boot-load-size 4 -boot-info-table -b boot/isolinux/isolinux.bin -c boot/isolinux/boot.cat -hide- rr-moved -o /path/to/bccd.iso /path/to/ISO_DIR

Advanced

If you were looking for information on the test suite, it's been moved here

Subversion Best Practices

See Subversion Best Practices for more information on how to use Subversion effectively for BCCD development.

build_livecd.pl and the Bccd.pm Perl module

Common variables

Common operations

The following are a list of operations that are common to the `build_livecd.pl` script. They are somewhat ordered, in that if you can get by with an earlier operation, you should.

Debian CLI

This should always be preferred to any of the other options. Use `apt-get`, `debootstrap`, `debconf`, etc. wherever possible, called from the system test in Bccd.pm. This might look like

$Bccd->run_test(
                "system",
                "",
                "Running apt-get install."
                "apt-get install $packages"
                );

Changing files

If you have to, change only part of the file using sed or perl regexen, Again, you can call these from the system tests in Bccd.pm. This might look like

$Bccd->run_test(
                "system",
                "",
                "Changing interface in networking."
                "sed -e 's/ifconfig eth0/ifconfig eth1/g' < \ 
$libdir/$projdir/etc/init.d/networking > $libdir/etc/init.d/networking"
                );

Entire file

You can store an entire file in subversion. Fetch it using the revfetch test, which takes the URL to the file, the subversion revison, and where you want to squirt the file in the build. This might look like

$Bccd->run_test(
              "revfetch",
              "",
              "Fetched checkin cron.",
              $svnrev,
              "$websvn/packages/etc/cron.d/node-checkin",
              "$builddir/etc/cron.d/node-checkin"
              );

Entire tree

Use this only if you are fetching a directory tree and not overwriting existing files. There are too many dependency problems involved with overwriting existing files in the build. chdir using the chdir test to the base directory of the destination directory tree. Then use the recrevfetch test, which takes a subversion revision, and a repository URL. This might look like

$Bccd->run_test(
              "chdir",
              "",
              "cd $builddir",
              "$builddir"
              );

$Bccd->run_test(
              "recrevfetch",
              "",
              "Fetched bin tools.",
              $svnrev,
              "$websvn/packages/bin/"
              );

Tar balls

The BCCD-NG development originally used tar balls to distribute changes. This is deprecated for a number of reasons, but where it's still working I haven't touched it. Do what I say and not what I do, and use the options above.

Init scripts

Init scripts are a SysV-based system's way of starting a service at a particular time. Init scripts in the BCCD are located either in packages/etc/init.d or trees/etc/init.d. Remember that files in packages need to be pulled down individually, while files in trees are pulled down en masse. Most init scripts need update-rc.d run, so most should end up in packages. Here's a step-by-step procedure for adding one:

  1. Add your init script to packages/etc/init.d in your SVN branch.
  2. Annotate the LSB block appropriately. For instance, below is the LSB block for packages/etc/init.d/networking. This makes sure that local filesystems are mounted (important for diskless nodes), and that the mountkernfs and bccd-nics init script have already run. It also says that, if available, ifupdown should be run before networking starts, and after it stops. See the Debian wiki for details on how LSB works in Debian.
    ### BEGIN INIT INFO
    # Provides:          networking
    # Required-Start:    mountkernfs $local_fs bccd-nics
    # Required-Stop:     $local_fs
    # Should-Start:      ifupdown
    # Should-Stop:       ifupdown
    # Default-Start:     2 5
    # Default-Stop:      0 1 6
    # Short-Description: Raise network interfaces.
    ### END INIT INFO
    
  3. In bin/build_livecd.pl, add a test to remove the init script's stop and start links. This is necessary so that they can be re-created with the proper dependency information. There are for-loops in build_livecd.pl, but a singleton test would look like this:
        $Bccd->run_test(
            "system",
            "",
            "Disabled $script.",
            "/usr/sbin/chroot $BUILDDIR /usr/sbin/update-rc.d -f some-script remove",
            );
    
  4. In bin/build_livecd.pl, add a test to re-addthe init script's stop and start links based on the new dependency information. There are for-loops in build_livecd.pl, but a singleton test would look like this:
        $Bccd->run_test(
            "system",
            "",
            "Disabled $script.",
            "/usr/sbin/chroot $BUILDDIR /usr/sbin/update-rc.d -f some-script defaults",
            );
    
  5. Commit everything to SVN.
  6. Rebuild the ISO.

Building the BCCD ISO

Changes should always be tested before being committed to trunk. This can be done on any Debian system with a checkout of the subversion repository.

  1. cd into branches/livecd-devel/bin/.
  2. script -fc "sudo perl build_livecd.pl --arch i386 --suite squeeze --outiso /tmp/test-$(whoami).iso --websvn (your branch in the web repository) " /tmp/build-script-$(whoami).log
    • Your branch should look something like this:
      • http://bccd-ng.cluster.earlham.edu/svn/bccd-ng/branches/(branch name)
  3. You can view the output of the test in /tmp/build-script-(your username).log.
    • try grep '^not ok' /tmp/build-script-$(whoami).log
    • If you have problems add --debug --nocleanup to the build_livecd.pl command line and append &> build-stderr-stdout.log to the command line.

Automated builds

After you get all the prerequisites setup and do a manual build, you can setup automated builds. First fetch auto_build_wrapper.sh from the bin directory of a branch or trunk.

  wget http://bccd-ng.cluster.earlham.edu/svn/bccd-ng/trunk/bin/auto_build_wrapper.sh && chmod +x auto_build_wrapper.sh

Then create configuration file that looks like this:

  ARCH=amd64
  DBUSER=dc_tester
  WEBSVN="http://bccd-ng.cluster.earlham.edu/svn/bccd-ng/"
  BRANCH="${WEBSVN}/trunk/"
  BINDIR="/bin/"
  BLDCD="build_livecd.pl"
  PATH=$PATH:/sbin:/usr/sbin
  OUTBASE=/cluster/bccd-ng/stable
  OUTISO=$OUTBASE/bccd_latest.${ARCH}.iso
  TEMPISO=$(mktemp ${OUTISO}.XXXXXXXX)

Then setup a cron job that looks like this:

  34 20 * * * /root/auto_build_wrapper.sh -c /root/auto_build_wrapper.i386.conf

Committing changes to trunk

  1. You'll need a working copy of trunk for this. If you haven't already, go to a folder and run svn co svn+ssh://leemasa@cluster.earlham.edu/cluster/svnroot/bccd-ng/trunk
    • Get comfortable, this will take a while.
    • If you have a working copy of trunk, make sure it's up-to-date with svn up .
  2. get into the root of your trunk directory
  3. Run svn merge --dry-run -r rev1:rev2 (your development working directory) .. rev1 is the last commit into the trunk, which you can see with svn log | less. rev2 is the new revision that you just committed to livecd-devel. Make sure the files being updated are the files you want updated. Look for any files with a "C" by them. You will have to merge those files by hand in the next step.
  4. svn merge -r rev1:rev2 svn+ssh://bccd-ng.cluster.earlham.edu/cluster/svnroot/bccd-ng/branches/livecd-devel/ .. Merge any conflicted files by hand, then run svn resolved filename. After everything is OK, run svn commit in the trunk directory, and commit your changes. Make sure that build_livecd.pl still has its websvn variable pointing into trunk and not livecd-devel.

Adding packages

See Reprepro for details on how the BCCD package repository works.

Via aptitude

aptitude is easy if there is an existing deb package in the tree on a0. Simply add it to the YAML configuration file bin/packages.conf. Most packages should be added to BASE. EXTRA is for packages that might become optional in the future, and AMD64 is for packages that should only be installed on AMD64 (rather than i386) builds.

Via svn

This option involves adding a "tree" to the subversion repository that contains the software built. Follow these steps to do it:

  1. From the live CD or liberation (liberation is recommended for disk space reasons), build and install the software into a subdirectory of /cluster/software.
  2. Make a symlink from software-xx.yy.zz to software, where xx.yy.zz is a version number.
  3. tar up software-xx.yy.zz and software, and move the tar ball to a host with a local subversion checkout.
  4. cd into the development branch, and then to trees/software/cluster/software.
  5. un-tar the tar ball.
  6. Run svn add software*.
  7. Remove the keywords property (this can cause binaries to become corrupted): find dir -type f ! -regex '.*\/\.svn\/.*' -print0|xargs -0 svn pd svn:keywords
  8. Commit the changes.
  9. Build a test ISO and test the new software.
  10. Merge the changes into the trunk.
  11. Document the process of building the package in the Software Build Documentation page.

Local builds

Using a local package mirror

By default, the build process uses the package mirror located at http://debmirror.cluster.earlham.edu/apt/mirror/bccd-ng/debian/. The script will allow the use of a local debmirror. Supply the hostname to the local repository using the --debmirror option. Don't supply the full URL; the rest of the URL will be constructed later in the script.

Using a local Subversion mirror

The build script fetches files out of Subversion using WebDAV at http://bccd-ng.cluster.earlham/svn/bccd-ng/branch-name, where branch-name would be trunch or branches/livecd-devel. You can supply an alternative WebDAV URL with --websvn.

Editing the splash screen

See the isolinux instructions for changing a boot screen: http://www.sweb.cz/Frantisek.Rysanek/splash/isolinux-splash-HOWTO.html

Kernel updates

See Kernel updates.

Rebuilds

You can make rebuilds faster by bypassing Subversion commits and just rebuilding the squashfs and KNOPPIX ISO images. To do this, do one full build with --nocleanup at the end, which will preserve the build directory in /tmp. Take a note of the directory that the script uses. Make your changes in the temporary directory (like $TMPDIR/build-bccd, or $TMPDIR/KNOPPIX). Then run

  rm -f $TMPDIR/KNOPPIX/KNOPPIX/KNOPPIX \
     && mksquashfs $TMPDIR/build-bccd $TMPDIR/KNOPPIX/KNOPPIX/KNOPPIX -comp xz \
     && genisoimage -pad -l -r -J -v -V BCCD-NG \ 
        -no-emul-boot -boot-load-size 4 -boot-info-table -b boot/isolinux/isolinux.bin \
        -c boot/isolinux/boot.cat -hide-rr-moved -o $OUTISO $TMPDIR/KNOPPIX

Environment management

As mentioned in the user manual, the BCCD uses Modules to manage a user's environment (variables, aliases, etc.). This provides a Tcl-based framework to manage a user's environment as software packages are added and deleted. Every package installed in /bccd/software should get a module directory in /usr/local/etc/modules, with every version of the package as a separate file in the directory. Every module file is a Tcl script invoked by the modules command. For example, OpenMPI version 1.4.0 has this file in /usr/local/etc/modules/openmpi/1.4.0 (source):

#%Module1.0#####################################################################

prereq          modules
conflict	lam
conflict	mpich2
set             MOD_OPENMPI              $env(BCCD_SW)/openmpi
set      MOD_OPENMPI_VERSION      1.4.0
set      MOD_OPENMPI_DIR          $MOD_OPENMPI-$MOD_OPENMPI_VERSION
module-whatis "OPENMPI"

prepend-path    PATH                $MOD_OPENMPI_DIR/bin/
prepend-path    LD_LIBRARY_PATH     $MOD_OPENMPI_DIR/lib/
prepend-path    MANPATH             $MOD_OPENMPI_DIR/share/man/
prepend-path    -d " " LDFLAGS      -L$MOD_OPENMPI_DIR/lib/
prepend-path    -d " " CPPFLAGS     -I$MOD_OPENMPI_DIR/include/

if { [catch { exec /bin/bccd-snarfhosts $env(HOME)/machines openmpi} msg] } {
	puts "bccd-snarfhosts failed: $::errorInfo"
}

These module files should be placed in trees/usr/local/etc/modules/software-name/version, which is part of the MODULEPATH search path. module load will load the lexigraphically last software version by default, but you can override this behavior by symlinking the version you want to be default to default.

USB automounting

The BCCD uses udev to detect when USB storage devices are plugged in, and mounts them automatically. The udev setup is in [1], and the script that the rule file calls is in [2].

SNMP

BCCD uses SNMP to gather information from other BCCD nodes. Its unique IANA private enterprise number is 40551.

Licensing & copyright

SVN properties

Todo - Document how to do this in ~/.subversion/config

In every file

Put this at the top of every file, in whatever comment blocks make sense:

  $Id$
  
  This file is part of BCCD, an open-source live CD for computational science
  education.
  
  Copyright (C) 2010 Andrew Fitz Gibbon, Sam Leeman-Munk, Charlie Peck,
     Skylar Thompson, & Aaron Weeden
  
  This program is free software: you can redistribute it and/or modify
  it under the terms of the GNU General Public License as published by
  the Free Software Foundation, either version 3 of the License, or
  (at your option) any later version.
  
  This program is distributed in the hope that it will be useful,
  but WITHOUT ANY WARRANTY; without even the implied warranty of
  MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
  GNU General Public License for more details.
  
  You should have received a copy of the GNU General Public License
  along with this program.  If not, see <http://www.gnu.org/licenses/>.
Personal tools
Namespaces
Variants
Actions
Navigation
Toolbox