User Tools

Site Tools


project:opennebula

This is an old revision of the document!


Home | Projects | SLUUG Hardware Refresh

OpenNebula on Xen on OpenSuSE 12.1

This is a working log - still in early draft. I'm still finalizing a lot of the config. If the formatting is confusing please disregard, I will be updating this as I make progress.

Create a VM on Xenpri1

Create VM on xenpri - virt-install -n nattylight -r 1024 –vcpu 1 –description "OpenNebula front-end, OpenSuSE 12.1 based" -l "http://download.opensuse.org/distribution/openSUSE-current/repo/oss/" –os-type linux –disk /etc/xen/images/nattylight.xvda,size=8 -w bridge=br0 –virt-type xen –arch x86_64

-x brokenmodules=edd xencons=tty

using Yast, or vm-install -

name:nattyice min-mem: 768 max-mem: 1024 vcpu: 2 description: OpenNebula front-end, openSuSE 12.1 based Graphics: none Virtual Disks: y Disk type: 2 (hard disk) Resides: /var/lib/xen/images/nattyice/xvda size: 8g Network adapters: add two Bootable media: 1 (network URL) Install URL: http://download.opensuse.org/distribution/openSUSE-current/repo/oss/

Enter through the rest

YaST2

Defaults unless otherwise stated Timezone: chicago Destkop: Other - Minimal Server Selection (Text Mode) Suggest Partitioning: uncheck Propose separate home, check Use btrfs Users: open-nebula-admin Password: ON

  • Do not use for sys admin, no auto login

Root PW: NI Enable Boot from MBR Enable SSH Software: install Patterns-openSUSE-webyast-{ui,ws}

Upon reboot - change hostname to nattyice

OpenNebula 3.1.0

  • Get the OpenSuSE x86_64 tarball
wget http://dev.opennebula.org/attachments/download/499/opennebula-3.1.0.tar.gz 
  • Install the packman repo
$ zypper ar ftp://anorien.csc.warwick.ac.uk/packman/suse/openSUSE_12.1/ Packman

Feel free to use a different mirror - http://packman.links2linux.de/mirrors

  • Add OpenNebula:Testing repo
 zypper ar http://download.opensuse.org/repositories/Virtualization:/Cloud:/OpenNebula:/Testing/openSUSE_12.1/ OpenNebula-Testing
  • Add OpenNebula (may need to do this after install…)
http://download.opensuse.org/repositories/Virtualization:/Cloud:/OpenNebula/openSUSE_12.1/

Install dependencies

 zypper install gcc gcc-c++ make patch libopenssl-devel libcurl-devel scons pkg-config sqlite3-devel libxslt-devel libxmlrpc_server_abyss++3 libxmlrpc_client++3 libexpat-devel libxmlrpc_server++3 libxml2-devel ruby ruby-doc-ri ruby-doc-html ruby-devel rubygems xmlrpc-c libmysqlclient-devel rubygem-rake libxmlrpc++3

Install via openSuSE repos

  • Add OpenNebula:Testing repo
 zypper ar http://download.opensuse.org/repositories/Virtualization:/Cloud:/OpenNebula:/Testing/openSUSE_12.1/ OpenNebula-Testing
  • Add OpenNebula (may need to do this after install…)
http://download.opensuse.org/repositories/Virtualization:/Cloud:/OpenNebula/openSUSE_12.1/
  • Install dependencies
 zypper install gcc gcc-c++ make patch libopenssl-devel libcurl-devel scons pkg-config sqlite3-devel libxslt-devel libxmlrpc_server_abyss++3 libxmlrpc_client++3 libexpat-devel libxmlrpc_server++3 libxml2-devel ruby ruby-doc-ri ruby-doc-html ruby-devel rubygems xmlrpc-c libmysqlclient-devel rubygem-rake libxmlrpc++3 mysql-community-server

MySQL install on openSuSE 12.1

  • Install packages
zypper install mysql-community-server
  • Start service
service mysql start
  • Secure SQL
mysql_secure_installation
  • Set root PW to the same as the root user PW, Select Y (defaults) for the rest
  • Create users with privileges
$ mysql -u root -p
Enter password: 
Welcome to the MySQL monitor. [...]

mysql> GRANT ALL PRIVILEGES ON opennebula.* TO 'oneadmin' IDENTIFIED BY 'oneadmin';
Query OK, 0 rows affected (0.00 sec)

Back to Open Nebula

  • openSuSE apparently forgets the install_gems script from OpenNebula - you'll need to manually install the following gems
gem install libxml-ruby json nokogiri rake xmlparser sqlite3 mysql curb thin uuid sinatra sequel amazon-ec2 
  • Install OpenNebula and Web UI (sunstone)
zypper install opennebula opennebula-sunstone

Install dependencies as needed - ignore the rubygem-* files we will install these later

  • Take note of onedmin user ID and cloud groupid
 id oneadmin 

In This case it's UID=1000 GID=1000

  • Logon to the xen host add a user of with the same UID/GID and a home of /var/lib/one
useradd --uid 100 -g cloud -d /var/lib/one oneadmin

If there is a conflict use usermod –uid UID to change the user who is in conflict

  • Set user's password to the same as the front-end

User's home dirs will eventually be moved to the NFS share for now copy the keys to the home dir of the xen box

  • Create ssh keypair for oneadmin, use defaults - no passphrase
ssh-keygen
  • Add key to auth users
$ cat ~/.ssh/id_rsa.pub >> ~/.ssh/authorized_keys
  • Change permissions of keys for to prevent issues later
$ chmod 700 ~/.ssh/
$ chmod 600 ~/.ssh/id_dsa.pub
$ chmod 600 ~/.ssh/id_dsa
$ chmod 600 ~/.ssh/authorized_keys
  • Create user ssh client config to prevent prompting for known hosts
vim ~/.ssh/config

Host *
    StrictHostKeyChecking no
  • Set OpenNebula to use the same credentials as oneadmin user
$ mkdir ~/.one
$ echo "oneadmin:password" > ~/.one/one_auth
$ chmod 600 ~/.one/one_auth
  • Start OneNebula == *Be sure to start as oneadmin user* ==
one start
  • Verify the oned is running and working
onevm list

Basic config

Enable MySQL
  • Config /etc/one/oned.conf (you can just hash the example)
DB = [ backend = "mysql",
       server  = "localhost",
       port    = 0,
       user    = "oneadmin",
       passwd  = "oneadmin",
       db_name = "opennebula" ]
Xen Host
  • Edit /etc/sudoers to allow oneadmin to control vms
visudo

oneadmin    ALL=(ALL) NOPASSWD: /usr/sbin/xm *
oneadmin    ALL=(ALL) NOPASSWD: /usr/sbin/xentop *
oneadmin    ALL=(ALL) NOPASSWD: /usr/sbin/ovs_vsctl *
  • Edit /etc/one/oned.conf to use Xen Driver (hash the kvm config example and unhash the xen example)
    IM_MAD = [
        name       = "im_xen",
        executable = "one_im_ssh",
        arguments  = "xen" ]

    VM_MAD = [
        name       = "vmm_xen",
        executable = "one_vmm_exec",
        arguments  = "xen",
        default    = "vmm_exec/vmm_exec_xen.conf",
        type       = "xen" ]
  • install ruby on xen host (>=1.8.7)
zypper install ruby
Storage
  • Map /etc/xen/images to btrfs raid 10 local array
  • Map /var/lib/one/ to NFS server 600G S/W raid 10
  • Map /home/* to NFS server 600G S/W raid 10
  • On NFS Server be sure to perform these fixes since root user access will be needed

-Set the permissions in the directory and the images exported to very permissive (directories: a+rx, files: a+rw) -Disable root squashing adding no_root_squash to nfs exporting options

/etc/exports/

/srv/storage/@/home     192.168.118.0/27(crossmnt,rw,no_root_squash,sync,no_subtree_check) 192.168.115.0/27(crossmnt,rw,no_root_squash,sync,no_subtree_check)
/srv/storage/@/backups  192.168.115.0/27(crossmnt,rw,no_root_squash,sync,no_subtree_check) 192.168.118.0/27(crossmnt,rw,no_root_squash,sync,no_subtree_check)
/var/lib/one    192.168.118.0/27(crossmnt,rw,no_root_squash,sync,no_subtree_check) 192.168.115.0/27(crossmnt,rw,no_root_squash,sync,no_subtree_check)
  • Recursive bind mount images data store. I for some reason am not getting crossmnt option to work on NFS
# mount --rbind /srv/storage/\@/images/ /var/lib/one/datastores/

/etc/fstab * This Doesn't work either, still experimenting… Probably need to mount device directly since its a subvolume *

/srv/storage/\@/images/ /var/lib/one/datastores/ none defaults,rbind
  • Edit /etc/one/oned.conf (default config)
TM_MAD = [
    name       = "tm_shared",
    executable = "one_tm",
    arguments  = "tm_shared/tm_shared.conf" ]
Networking - Open vSwitch
  • Edit /etc/one/oned.conf (unhash open vSwitch VM_HOOK)
VM_HOOK = [
    name      = "openvswitch-vlan",
    on        = "RUNNING",
    command   = "vnm/openvswitch-vlan",
    arguments = "$TEMPLATE",
    remote    = "yes" ]

**** INSTALL ON HOST(S) ****

  • Install Open vSwitch from source since it has failed openSuSE build project
  • Install dependencies
 zypper install python-qt4 python-Twisted python-zope.interface tcpdump kernel-xen-base kernel-xen-devel autoconf automake dotconf gcc glibc libopenssl-devel make

Packages for Debian: python-qt4 python-twisted python-zopeinterface tcpdump linux-headers autoconf automake libdotconf-dev libdotconf1.0 gcc glibc-source libc-dev libssl-dev <sub>Switch to build user

  • Get tarball
wget http://openvswitch.org/releases/openvswitch-1.2.2.tar.gz
  • Untar, change to dir
tar -xzf openvswitch-1.2.2.tar.gz
	cd openvswitch...
  • Configure with unique options (at this time openvswitch will not compile kernel modules for kernel 3.1)
./configure --prefix=/usr --localstatedir=/var 
  • Make, then su make install
$make
#make install
  • Intialize configuration database
#ovsdb-tool create /usr/etc/openvswitch/conf.db vswitchd/vswitch.ovsschema
  • Initialize configuration database server
ovsdb-server --remote=punix:/var/run/openvswitch/db.sock \
                     --remote=db:Open_vSwitch,manager_options \
                     --private-key=db:SSL,private_key \
                     --certificate=db:SSL,certificate \
                     --bootstrap-ca-cert=db:SSL,ca_cert \
                     --pidfile --detach
  • initialize database
 ovs-vsctl ---no-wait init
  • Start the Open vSwitch daemon
ovs-vswitchd --pidfile --detach

Note there are a few errors that we should be able to safely diregard since we did not compile the kernel module

* need kernel modules for brctl layer

vSwitch doesn't like kernel 3.1 - we'll likely need to use ebtables which will conflict with xenpri0

Work In Progress - 21DEC11

project/opennebula.1339551652.txt.gz · Last modified: 2012/06/12 20:40 by SLUUG Administration