Configuration of Host Hardware and OS for Coordinating Nodes
============================================================

This document describes the setup of a virtual machine host on which will
operate a DataONE Coordinating Node. The steps take for setup and
configuration up to the point where a clean Ubuntu client is installed are
described.


Hardware
--------

.. table::
   Specifications of the hardware being installed for the UNM node.

   ================== ===============================================
   Attribute          Value
   ================== ===============================================
   Brand, Model       Dell, R905
   Processors         4x Quad Core Opteron 8378 2.4GHz (Shanghai), 
                      4x512K L2 Cache, 6MB L3 Cache, HyperTransport 1Ghz
   Memory             64GB 800MHZ ECC (16X4GB)
   Disks              5x 600GB 10K RPM Serial-Attach SCSI 3Gbps 3.5-in 
                      HotPlug HardDrive
   Disk Controller    PERC6i SAS RAID Controller Internal with Battery
   NICs               4x Broadcom NetXtreme II 5708 1GbE Onboard NICs 
                      with TOE.  Configured with LAG enabled.
   ================== ===============================================

.. table::
   Specifications of the hardware being installed for the UCSB node.

   ================== ===============================================
   Attribute          Value
   ================== ===============================================
   Brand, Model       Dell, R905
   Processors         4x Quad Core Opteron 8380 2.5GHz (Shanghai),
                      4x512K L2 Cache, 6MB L3 Cache, HyperTransport 1Ghz
   Memory             64GB 800MHZ ECC (16X4GB)
   Disks              5x 600GB 15K RPM Serial-Attach SCSI 3Gbps 3.5-in 
                      HotPlug HardDrive
   Disk Controller    PERC6i SAS RAID Controller Internal with Battery
   NICs               4x Broadcom NetXtreme II 5708 1GbE Onboard NICs 
                      with TOE.  Configured with LAG enabled.
   ================== ===============================================



Disk Configuration
~~~~~~~~~~~~~~~~~~

Disks are configured as RAID 1+0 (mirror + striping) with one hot spare,
providing 1.2TB.


OS Installation
~~~~~~~~~~~~~~~

Boot from Ubuntu 9.10 64 bit server distribution.  

.. Note:: The OS was upgraded to 10.04 on 2010-05-10 with no apparent side
          effects besides the need to set the correct host name (reverted 
          to "ubuntu")

.. table::
   OS installation configuration values at UNM

   =============== =================================
   Attribute       Value
   =============== =================================
   OS              Ubuntu 10.04 64 bit server
   Server name     host-unm-1
   Server FQDN     host-unm-1.dataone.org
   IP Address      129.24.0.18
   Netmask         255.255.255.0
   Gateway         129.24.0.1
   DNS 1           129.24.0.52
   DNS 2           
   Partitions      swap 16GB
                   LVM everything else.
   =============== =================================


.. table::
   OS installation configuration values at UCSB

   =============== =================================
   Attribute       Value
   =============== =================================
   OS              Ubuntu 10.04 64 bit server
   Server name     host-ucsb-1
   Server FQDN     host-ucsb-1.dataone.org
   IP Address      128.111.220.52
   Netmask         255.255.255.0
   Gateway         128.111.220.1
   DNS 1           128.111.220.18
   DNS 2           128.111.220.16
   Partitions      /boot 240MB
                   LVM everything else.
   =============== =================================


.. table::
   OS installation configuration values at ORC
   
   =============== =================================
   Attribute       Value
   =============== =================================
   OS              Ubuntu 10.04 64 bit server
   Server name     orcvm1
   Server FQDN     orcvm1.dataone.org. host-orc-1.dataone.org
   IP Address      160.36.134.68
   Netmask         255.255.255.240
   Gateway         160.36.234.65
   DNS 1           160.36.134.65
   Partitions      /boot 240MB
                   LVM everything else.
   =============== =================================



Partition setup at UCSB

- during install, select "Guided - use entire disk and setup LVM", then enter
  "40 GB" when asked how much space to use

- after install extend swap and make partition for kvm::

  $ sudo lvextend /dev/host-ucsb-1/swap_1 -L 16G
  $ sudo lvcreate -l 100%FREE -n kvm host-ucsb-1
  $ sudo swapoff -a
  $ sudo mkswap /dev/host-ucsb-1/swap_1
  $ sudo swapon -a
  $ sudo mkfs.ext4 /dev/host-ucsb-1/kvm

- kvm partition mounted at the default ubuntu location /var/lib/libvirt/images
  and symlinked to /kvm, so as to not need to change /etc/libvirt config files


The partitions (UNM)::

  # parted -l
  Model: DELL PERC 6/i Adapter (scsi)
  Disk /dev/sda: 1199GB
  Sector size (logical/physical): 512B/512B
  Partition Table: msdos

  Number  Start   End     Size    Type      File system  Flags
   1      32.3kB  1199GB  1199GB  primary                boot, lvm
   2      1199GB  1199GB  255MB   extended
   5      1199GB  1199GB  255MB   logical   ext2


  Model: Linux device-mapper (linear) (dm)
  Disk /dev/mapper/dataone-kvm: 1103GB
  Sector size (logical/physical): 512B/512B
  Partition Table: loop

  Number  Start  End     Size    File system  Flags
   1      0.00B  1103GB  1103GB  ext4


  Model: Linux device-mapper (linear) (dm)
  Disk /dev/mapper/dataone-home: 40.0GB
  Sector size (logical/physical): 512B/512B
  Partition Table: loop

  Number  Start  End     Size    File system  Flags
   1      0.00B  40.0GB  40.0GB  ext4


  Model: Linux device-mapper (linear) (dm)
  Disk /dev/mapper/dataone-root: 40.0GB
  Sector size (logical/physical): 512B/512B
  Partition Table: loop

  Number  Start  End     Size    File system  Flags
   1      0.00B  40.0GB  40.0GB  ext4


  Model: Linux device-mapper (linear) (dm)
  Disk /dev/mapper/dataone-swap: 16.0GB
  Sector size (logical/physical): 512B/512B
  Partition Table: loop

  Number  Start  End     Size    File system     Flags
   1      0.00B  16.0GB  16.0GB  linux-swap(v1)



After the OS installation, setup for running KVM guests. Detailed instructions
are available at the `Ubuntu site`_ ::

  $ sudo apt-get install kvm libvirt-bin ubuntu-vm-builder bridge-utils
  $ sudo adduser `id -un` libvirtd


.. _Ubuntu site: https://help.ubuntu.com/community/KVM


Edit ``/etc/network/interfaces`` to support bridging so the KVM guests can use
external IP addresses::

  $ cat /etc/network/interfaces
  # This file describes the network interfaces available on your system
  # and how to activate them. For more information, see interfaces(5).

  # The loopback network interface
  auto lo
  iface lo inet loopback

  # The primary network interface
  auto eth0
  iface eth0 inet manual

  # The bridge network for KVM
  auto br0
  iface br0 inet static
  	address 129.24.0.18
  	netmask 255.255.255.0
  	network 129.24.0.0
  	broadcast 129.24.0.255
  	gateway 129.24.0.1
          bridge_ports eth0
          bridge_stp off
          bridge_fd 0
          bridge_maxwait 0

  	# dns-* options are implemented by the resolvconf package, if installed
  	dns-nameservers 129.24.0.52
  	dns-search nmepscor.net


Modify the default configuration for ufw_ to enable forwarding by editing
``/etc/default/ufw`` and set::

  DEFAULT_FORWARD_POLICY="ACCEPT"
  
Then enable ssh and start the firewall::

  sudo ufw allow ssh
  sudo ufw enable


Network config at UCSB with two LACP bonded ports on an LACP enabled switch
with bridging::

  $ cat /etc/network/interfaces
  # This file describes the network interfaces available on your system
  # and how to activate them. For more information, see interfaces(5).
  
  # The loopback network interface
  auto lo
  iface lo inet loopback
  
  # The primary network interface
  auto bond0
  iface bond0 inet manual
          bond-slaves none
          bond-mode 4
          bond-miimon 100
  
  auto eth0
  iface eth0 inet manual
          bond-master bond0
          bond-primary eth0 eth1
  
  auto eth1
  iface eth1 inet manual
          bond-master bond0
          bond-primary eth0 eth1
  
  auto br0
  iface br0 inet static
          address 128.111.220.52
          netmask 255.255.255.0
          gateway 128.111.220.1
          broadcast 128.111.220.255
          bridge_ports bond0
          bridge_fd 0
          bridge_maxwait 0
  

.. _ufw: https://help.ubuntu.com/10.04/serverguide/C/firewall.html