Jenkins

From BCCD 3.0

Jump to: navigation, search

Contents

Introduction

Jenkins is a continuous integration system that the BCCD project uses for automating building and testing of BCCD.

Access

Our local Jenkins server may be found at https://bigfe.cluster.earlham.edu/jenkins/. Login using your CCG username/password.

Note that bigfe is not accessible outside the CCG network (159.28.23.0/24). You may access it from outside using SSH SOCKS proxying. Use your cluster LDAP username/password to login.

  1. Place this block in your ~/.ssh/config:
     Host hopper
             HostName hopper.cluster.earlham.edu
             DynamicForward 1081
    
  2. ssh hopper (not hopper.cluster.earlham.edu!)
  3. In your browser proxy configuration (for Firefox this is Preferences->Advanced->Network Settings) set your SOCKS v5 proxy to be localhost, port 1081.
  4. You should now be able to access https://bigfe.cluster.earlham.edu/jenkins/

Builds

Builds are automated processes that Jenkins uses to create a software product.

Creating a new build

You will want to do this after you create a new SVN branch.

  1. In the branch, make a bin/build_livecd.conf file that looks like this, changing the parameters as needed.
     SUITE   :   squeeze
     OUTDIR  :   /cluster/bccd-ng/testing/skylar
     WEBSVN  :   http://bccd-ng.cluster.earlham.edu/svn/bccd-ng/branches/skylar/bccd-3.3.2
     RELEASE :   3.3.2-skylar
    
  2. Click "New Item"
  3. Give the build a name based on the branch name. Avoid the use of spaces in the name, as it will be used in the workspace directory name and not all utilities (i.e. debootstrap) properly deal with paths with spaces.
  4. Select "Build a free-style software project"
  5. Select Subversion under "Source Code Management".
  6. Supply the SVN repo URL for the branch you want to build (i.e. http://cluster.earlham.edu/svn/bccd-ng/branches/skylar/bccd-3.3.2)
  7. Click on "Add build step" and select "Execute Shell".
  8. Enter a variation of this, making sure to change the architecture as appropriate (choices are i386 or amd64):
    PERL5LIB=./trees/usr/local/lib/site_perl /usr/bin/perl bin/build_livecd.pl --arch i386
    

Cloning a build

  1. Click "New Item"
  2. Give the build a name based on the branch name.
  3. Select "Copy existing item"
  4. Enter existing item name

Scheduling a build

From the Jenkins home page (aka build dashboard), click the icon on the far right.

Troubleshooting a build

A failed build will be indicated by a red orb. A project with repeatedly-failed builds will have a thundercloud by it.

Diagnosing a failed build generally involves looking at the console output:

  1. From the Jenkins login page, go to Build History
  2. Click on the terminal icon associated with the failed build.

One can also see the "workspace" of the build, which contains all the files and directories used.


Install

For bigfe:

  1. Make sure you do not have Java 6: apt-get remove openjdk-6-jre{,-{lib,headless}}
  2. Make sure you have openjdk-7-jre installed: apt-get install openjdk-7-jre
    • If it doesn't exist, link /usr/lib/jvm/java-1.7.0-openjdk-amd64 to /usr/lib/jvm/default-java
  3. Follow the Debian/Ubuntu install guide. This will give you a Jenkins daemon running in the background.
  4. In /etc/default/jenkins:
    • Set JENKINS_USER=root. This is needed to have proper ownership as files are copied into the build directory.
    • Set AJP_PORT=8009
    • Uncomment JAVA_ARGS="-Djava.net.preferIPv4Stack=true" to force IPv6
    • Add --prefix=$PREFIX to JENKINS_ARGS
  5. Start Jenkins: invoke-rc.d jenkins start
  6. Setup SSH SOCKS proxy through hopper.
  7. Go to http://bigfe.cluster.earlham.edu:8080
    1. Go to Manage Jenkins
    2. Enable Security
    3. Set Access Control to LDAP
      • Server to cluster.earlham.edu
    4. Advanced options: Root DN - dc=cluster,dc=loc
    5. Enable Cross Site Request protection w/ default crumbs
  8. Apache2
    1. Install libapache2-mod-jk
    2. Symlink the following files from /etc/apache2/mods-available to /etc/apache2/mods-enabled:
      • ssl.load
      • jk.load
      • rewrite.load
    3. Set the following in /etc/libapache2-mod-jk/workers.properties
      worker.list=jenkins,jk-status,jk-manager
      worker.jenkins.port=8009
      worker.jenkins.host=localhost
      worker.jenkins.type=ajp13
      # configure jk-status
      worker.list=jk-status
      worker.jk-status.type=status
      worker.jk-status.read_only=true
      # configure jk-manager
      worker.list=jk-manager
      worker.jk-manager.type=status
      
    4. Add the following to /etc/apache2/conf.d/jenkins:
      JkLogFile     /var/log/apache2/mod_jk.log
      JkLogLevel    info
      JkLogStampFormat "[%a %b %d %H:%M:%S %Y] "
      JkOptions     +ForwardKeySize +ForwardURICompat -ForwardDirectories
      JkRequestLogFormat     "%w %V %T"
      
    5. Add the following to /etc/apache2/sites-available/default-ssl
      JkMount  /jenkins/* jenkins
      
    6. Symlink /etc/apache2/sites-available/default-ssl to /etc/apache2/sites-enabled/001-default-ssl
    7. Add the following to /etc/apache2/sites-available/default to force SSL use for Jenkins:
      # Force SSL for Jenkins
      <Location /jenkins>
         RewriteEngine on
         RewriteCond %{HTTPS} off
         RewriteRule ^/?(.*) https://%{SERVER_NAME}%{REQUEST_URI}/ [L,R]
      </Location>
      
  9. Setup a firewall to protect the service
    1. Install the iptables-persistent package if it is not already installed
    2. Agree to save current rule set (both IPv4 and IPv6)
    3. Add this line to /etc/iptables/rules.v4 before the COMMIT:
      # Block all Jenkins connections not from 127.0.0.1
      -A INPUT -p tcp ! -s 127.0.0.1 --dport 8080 -j REJECT
      
    4. Add this line to /etc/iptables/rules.v6 before the COMMIT:
      # Block all Jenkins connections not from 127.0.0.1
      -A INPUT -p tcp ! -s ::1 --dport 8080 -j REJECT
      
    5. Run invoke-rc.d iptables-persistent stop && invoke-rc.d iptables-persistent start
    6. Run iptables -L and ip6tables -L and verify that the new rules are in place.
Personal tools
Namespaces
Variants
Actions
Navigation
Toolbox