From BCCD 3.0

Revision as of 00:37, 13 May 2022 (view source) Skylar (Talk | contribs) ← Older edit		Latest revision as of 00:47, 13 May 2022 (view source) Skylar (Talk | contribs)
Line 193:		Line 193:
	dhcp-range=10.0.0.100,10.0.0.200,12h		dhcp-range=10.0.0.100,10.0.0.200,12h
	</pre></li>		</pre></li>
		+	<li>Initialize vagrant environment: <code>vagrant init debian/stretch64</code></li>
		+	<li>Start: <code>VAGRANT_HOME=/var/tmp vagrant up --provider=lxc</code></li>
	</ol>		</ol>

---

Latest revision as of 00:47, 13 May 2022

Contents 1 General commentary 2 Re-working using custom scripts 2.1 Proposal 2.1.1 Jenkins integration 2.1.2 Makefile 2.1.3 Ownership/permissions tracking 2.1.4 File conflicts b/w packages 2.1.5 Out-of-date configurations 2.1.6 deb scripts 2.1.7 Pinning packages to BCCD repo 2.2 Packaging notes 2.2.1 dch 2.2.2 dh_make 2.2.3 debuild 2.2.4 dpkg-buildpackage 2.2.5 lintian 3 live-build 3.1 lb config 3.2 lb build 3.3 Logging 3.4 Vagrant/LXC + Jenkins

General commentary

• DEBMIRROR
  • Why is it necessary to allow a hard-coded setting for debmirror when we have a configuration file?
• HARD-CODED SETTINGS
  • Can any/all of these be moved into the configuration file?
• FILE COPIES
  • Can these be packaged up?
• Documentation
  • Can we ship man pages?

Re-working using custom scripts

BCCD is a Debian distribution with some customizations:

• Custom kernel for stackable filesystem (aufs)
• Lots of custom files that are pulled from SVN during ISO build
• Two custom run modes:
  • Live (transient, off ISO)
  • Liberated (persistent, off HDD)

Proposal

1. Write a build driver that produces a stock Debian live ISO w/ custom kernel for aufs
2. Package up the contents of SVN as one or more deb packages
3. Install deb packages as part of build

Jenkins integration

The following Jenkins projects will be used:

fpm 

Build the .deb package with fpm

iso 

Build the ISO image after fpm, using debootstrap

Makefile

Jenkins can call Makefile targets:

target/bccd.noarch.deb 

Builds the .deb package, triggers ISO build

target/bccd.amd64.iso 

Builds the ISO image in a temporary directory

• • Requires that a fresh checkout (or simulation) be done so the temporary directory is removed

debootstrap 

Bootstraps a chroot'd Debian system, removes directory if it fails

Ownership/permissions tracking

Assign the following SVN attributes to files:

bccd:username 

User name for file after packaging

bccd:groupname 

Group name for file after packaging

bccd:permmode 

Permissions mode for file after packaging

The process that builds the deb package will produce a YAML dictionary of dictionaries, keyed on file name. Each key will then point to a second dictionary, with entries for the above attributes.

A post-install script can set the permissions and ownership for each file based on the array.

File conflicts b/w packages

BCCD will install files that conflict with other packages. While it does its best to avoid the situation via include files or .d directoriews, some packages do not support this (i.e. /etc/dhcp/dhclient.conf).

The legacy build process simply overwrites files, but our goal here is to package all files. To that goal, look into using diversions:

• https://wiki.debian.org/Adding%20and%20removing%20diversions
• https://debian-administration.org/article/118/Replacing_binaries_with_dpkg-divert
• https://www.debian.org/doc/debian-policy/ap-pkg-diversions.html

In testing, the --local causes conflicts even if the file is renamed. Running just with --rename solves the problem.

A for loop in the before-install script will have a list of conflicting files and generate a list of files to divert, which can then be used by the after-install script to remove the diversions.

Out-of-date configurations

These are configurations that can be removed:

bccd-sshd 

This used to be used to generate host SSH keys. Presumably at some point this didn't happen automatically, but it does now

deb scripts

Scripts executed by the deb package should be placed in bin/deb and referenced in the Makefile via fpm.

Pinning packages to BCCD repo

The /etc/apt/preferences.d/100_bccd file will set a high (900) priority for all packages that can be found in the BCCD apt repo (keying on Origin in Reprepro).

See https://wiki.debian.org/AptPreferences for more details.

Packaging notes

Probably not anything useful for us, kept for posterity

Links:

• https://www.debian.org/doc/manuals/maint-guide/first.en.html
• https://wiki.debian.org/Packaging
• https://www.debian.org/doc/manuals/maint-guide/start.en.html
• https://beingasysadmin.wordpress.com/2014/12/07/automating-debian-package-management/

Questions:

• How to maintain version control?
• How to continue using CI?

dch

Used to maintain the debian/changelog file.

• --create - Create a new changelog file
• --append - Add changelog entry
• --increment - Increment version number
• --edit - Interactive editor mode
• --release - Finalize changelog on release
• --closes - Closes particular bug

dh_make

• Depends on various environment variables:

  DEBEMAIL="your.email.address@example.org"
  DEBFULLNAME="Firstname Lastname"
  export DEBEMAIL DEBFULLNAME
  

debuild

dpkg-buildpackage

lintian

live-build

• live-build seems to be pretty immature; I (and a number of other people) have been unable to get it to work on modern Debian or Ubuntu

• Overview
• Current test

lb config

• lb config (aka lb_config(1)) makes a config skeleton. Useful options:
  • --architecture: Specify ISA (i.e. amd64)
  • --binary-images: Specify output format (i.e. iso)
  • --apt: Set to apt or aptitude (probably aptitude)
  • --bootstrap-flavour: Minimal or standard, probably want minimal so we can select our own packages *this option does not appear to exist in the wheezy build*
  • --distribution: Set to current Debian codename
  • --interactive: Specify interactive shell to use for testing purposes
  • --archive-areas: Specify archive areas of a package repo to include
  • --linux-packages: Supply custom linux .deb package name
  • --apt-indices: Set to false to remove apt indices after build
  • --apt-recommends: Set to false to install only prioritized packages
  • --apt-options / --aptitude-options: Pass in apt/aptitude options directly
  • --bootappend-live "boot=live components username=live-user": Customize live user name
• Provide custom package lists in config/chroot_local-packageslists
  • #if pragma can match distribution, architecture, and archive area
• live/chroot includes can provide files directly in live image w/o packaging them
• hook scripts can be run after applying configuration
• live user made dynamically at boot time

lb build

lb build will start the build process.

Logging

• Use Python logging module

Vagrant/LXC + Jenkins

The goal is to isolate the build host from the build process. See https://wiki.debian.org/VagrantLXC

1. Install vagrant and vagrant-lxc packages
2. Setup /etc/default/lxc-net and then restart lxc-net service:

   USE_LXC_BRIDGE="true"
   LXC_BRIDGE="lxcbr0"
   LXC_ADDR="10.0.3.1"
   LXC_NETMASK="255.255.255.0"
   LXC_NETWORK="10.0.3.0/24"
   LXC_DHCP_RANGE="10.0.3.2,10.0.3.254"
   LXC_DHCP_MAX="253"
   LXC_DHCP_CONFILE=""
   LXC_DOMAIN=""
   

3. Setup /etc/lxc/default.conf:

   lxc.network.type = veth
   lxc.network.flags = up
   lxc.network.link = lxc-bridge-nat
   

4. Setup /etc/dnsmasq and then restart dnsmasq service:

   interface=lxc-bridge-nat
   dhcp-range=10.0.0.100,10.0.0.200,12h
   

5. Initialize vagrant environment: vagrant init debian/stretch64
6. Start: VAGRANT_HOME=/var/tmp vagrant up --provider=lxc