eucalyptus2.0.1-src-install in CentOS5.5 x86(原)
The CloudController (CLC), Walrus (W) and Storage Controller (SC) are written in Java,
while the ClusterController (CC) and the Node Controller (NC) are written in C
Front-end run-timedependencies
Java 6 Make sure that yourJAVA_HOME environment variable is set to the location of your JDK.
must run a serveron port 25: that can deliver or relay email messages to cloud users' emailaddresses. To test whether you have a properly functioning mail relay forlocalhost, try to send email to yourself from the terminal using"mail"
For MANAGED:
iproute and iptables packages (ip andiptables commands must work)
DHCP Server compatible with ISC DHCPDaemon version 3.0.X (dhcp3-server)
bridge-utils package (brctl command mustwork)
vtun package, for multi-clusterconfigurations
vlan package (vconfig command must work)
For EBS:
lvm2 package (e.g., command lvm shouldwork)
aoetools package
vblade package
Node run-timedependencies
Perlscripts
Two hypervisors(xen & kvm) aresupported
FOR MANAGED:
iproute and iptables packages(ip and iptables commands must work)
bridge-utils package (brctlcommand must work)
vlan package (vconfig commandmust work)
libvirt package (potentially withlibvirtd, depending on hypervisor configuration)
You must be rootto install and start Eucalyptus components
The version ofOpenJDK that is bundled with CentOS-5 cannot compile the version of GWT thatcomes with Eucalyptus as a dependency. You will need to install JDK 1.6.0"manually". We use Sun's JDK, which can be found at http://java.sun.com/javase/downloads/index.jsp.Be sure to set your JAVA_HOME and PATH properly before running the Eucalyptus'configure' script.
yum install -y java-1.6.0-openjdk-devel ant ant-nodeps libvirt-devel curl-devel httpd httpd-devel apr-devel openssl-devel dhcp libxml2 libxml2-devel gnutls gnutls-devel xen-devel libgcrypt-devel zlib-devel perl-Convert-ASN1 perl-Crypt-OpenSSL-RSA perl-Crypt-OpenSSL-Random chkfontpath scsi-target-utils fuse-libs swig gcc
2. Download Eucalyptus and supplied dependencies
Download either
- eucalyptus-$VERSION-src.tar.gz (Eucalyptus source with included java libraries)
or
- eucalyptus-$VERSION-src-online.tar.gz (Eucalyptus source that will download java libraries at build-time)
and for both
- eucalyptus-$VERSION-src-deps.tar.gz (Eucalyptus C library dependency packages)
All packages can be found on the Eucalyptus Web site:
Unpack the Eucalyptus source:
tar zvxf eucalyptus-$VERSION-src.tar.gz
Now you should have a directory eucalyptus-$VERSION. Tosimplify the remainder of the installation, define EUCALYPTUS_SRC environmentvariable to be the top of the source tree of eucalyptus and the variableEUCALYPTUS to be the directory where eucalyptus will be installed (we recommendusing /opt/eucalyptus/):
cd eucalyptus-$VERSION
export EUCALYPTUS_SRC=`pwd`
export EUCALYPTUS=/opt/eucalyptus
3. Build Dependencies
To install Eucalyptus, you need to build packages thatEucalyptus depends on, which we provide in the above-mentioned packageeucalyptus-$VERSION-src-deps.tar.gz. For the sake of this discussion, we aregoing to assume that all packages have been untarred inside "$EUCALYPTUS_SRC/eucalyptus-src-deps/"as above and will be installed in "$EUCALYPTUS/packages".
Unpack the dependencies and create the directory you'll useto install them:
cd $EUCALYPTUS_SRC
tar zvxf ../eucalyptus-$VERSION-src-deps.tar.gz
mkdir -p $EUCALYPTUS/packages/
Build and install the dependencies.
a. Axis2
cd $EUCALYPTUS/packages
tar zxvf $EUCALYPTUS_SRC/eucalyptus-src-deps/axis2-1.4.tgz
b. Axis2/C
To compile Axis2/C, you will need to locate developmentheaders for Apache and for APR.
On CentOS 5, the headers should be in the followinglocation:
在这之前附加：需要安装一些库文件，如下：
rpm -Uvh httpd-devel-2.2.3-43.el5.centos.x86_64.rpmapr-devel-1.2.7-11.el5_3.1.x86_64.rpm apr-util-devel-1.2.7-11.el5.x86_64.rpmdb4-devel-4.3.29-10.el5.x86_64.rpm expat-devel-1.95.8-8.3.el5_4.2.x86_64.rpmopenldap-devel-2.3.43-12.el5.x86_64.rpmcyrus-sasl-devel-2.1.22-5.el5_4.3.x86_64.rpm
export APACHE_INCLUDES=/usr/include/httpd/
export APR_INCLUDES=/usr/include/apr-1/
With the two environment variables set, you can build andinstall Axis2/C as follows:
export AXIS2C_HOME=$EUCALYPTUS/packages/axis2c-1.6.0
cd $EUCALYPTUS_SRC/eucalyptus-src-deps/
tar zvxf axis2c-src-1.6.0.tar.gz
cd axis2c-src-1.6.0
CFLAGS="-w" ./configure --prefix=${AXIS2C_HOME} --with-apache2=$APACHE_INCLUDES --with-apr=$APR_INCLUDES --enable-multi-thread=no
make ; make install
c. Rampart/C
附加：编译安装前，需安装一些库文件：
rpm -Uvh openssl-devel-0.9.8e-12.el5_4.6.x86_64.rpmkrb5-devel-1.6.1-36.el5_4.1.x86_64.rpm zlib-devel-1.2.3-3.x86_64.rpme2fsprogs-devel-1.39-23.el5.x86_64.rpm keyutils-libs-devel-1.2-1.el5.x86_64.rpmlibselinux-devel-1.33.4-5.5.el5.x86_64.rpmlibsepol-devel-1.15.2-3.el5.x86_64.rpm
export AXIS2C_HOME=$EUCALYPTUS/packages/axis2c-1.6.0
export LD_LIBRARY_PATH=${AXIS2C_HOME}/lib:$LD_LIBRARY_PATH
cd $EUCALYPTUS_SRC/eucalyptus-src-deps/
tar zvxf rampartc-src-1.3.0-0euca1.tar.gz
cd rampartc-src-1.3.0
./configure --prefix=${AXIS2C_HOME} --enable-static=no --with-axis2=${AXIS2C_HOME}/include/axis2-1.6.0
make ; make install
Next, change the following in $AXIS2C_HOME/axis2.xml. In the 'inflow' section,change:
<!--phase name="Security"/-->
to
<phase name="Security"/>
In the 'outflow' section, change:
<!--phase name="Security"/-->
to
<phase name="Security"/>
4. Building Eucalyptus
First, make sure JAVA_HOME is defined. For example, onCentos 5:
export JAVA_HOME="/usr/lib/jvm/java-openjdk/"
export JAVA="$JAVA_HOME/jre/bin/java"
实际配置如下：
export JAVA_HOME=/usr/lib/jvm/java-1.6.0-openjdk.x86_64
export JAVA=$JAVA_HOME/jre/bin/java
then, build Eucalyptus:
附加：安装之前需要安装一些库文件(如下命令运行，并不能正常安装，为何？)
rpm -Uvh xen-devel-3.0.3-105.el5.x86_64.rpmlibvirt-0.6.3-33.el5.x86_64.rpm libvirt-devel-0.6.3-33.el5.x86_64.rpm
rpm -Uvh curl-devel-7.15.5-9.el5.x86_64.rpmlibidn-devel-0.6.5-1.1.x86_64.rpm
// 或者
rpm -Uvh kernel-xen-2.6.18-194.el5.x86_64.rpmkernel-xen-devel-2.6.18-194.el5.x86_64.rpmpython-virtinst-0.400.3-9.el5.noarch.rpmxz-libs-4.999.9-0.3.beta.20091007git.el5.x86_64.rpme4fsprogs-libs-1.41.9-3.el5.x86_64.rpm xen-libs-3.0.3-105.el5.x86_64.rpmxen-3.0.3-105.el5.x86_64.rpm xen-devel-3.0.3-105.el5.x86_64.rpm libvirt-0.6.3-33.el5.x86_64.rpmlibvirt-devel-0.6.3-33.el5.x86_64.rpm libvirt-python-0.6.3-33.el5.x86_64.rpmiscsi-initiator-utils-6.2.0.871-0.16.el5.x86_64.rpmgpxe-roms-qemu-0.9.7-8.el5.x86_64.rpm kvm-qemu-img-83-164.el5.x86_64.rpmcelt051-0.5.1.3-0.el5.x86_64.rpm log4cpp-1.0-9.el5.x86_64.rpm qffmpeg-libs-0.4.9-0.15.20080908.el5.x86_64.rpmqcairo-1.8.7.1-3.el5.x86_64.rpm qpixman-0.13.3-4.el5.x86_64.rpmtclspice-21-3.cvs20100719.el5.x86_64.rpm
rpm –ivh tclspice-21-3.cvs20100719.el5.x86_64.rpm
// 安装ant
rpm -ivh ant-1.6.5-2jpp.2.x86_64.rpmjava-1.4.2-gcj-compat-1.4.2.0-40jpp.115.x86_64.rpmlibgcj-4.1.2-48.el5.x86_64.rpm xml-commons-apis-1.3.02-0.b2.7jpp.10.x86_64.rpmgjdoc-0.7.7-12.el5.x86_64.rpm xml-commons-1.3.02-0.b2.7jpp.10.x86_64.rpmantlr-2.7.6-4jpp.2.x86_64.rpm
// 安装ant-noteps
rpm -ivh dhcp-3.0.5-23.el5.x86_64.rpm
cd $EUCALYPTUS_SRC
./configure --with-axis2=$EUCALYPTUS/packages/axis2-1.4 --with-axis2c=$EUCALYPTUS/packages/axis2c-1.6.0 --enable-debug --prefix=$EUCALYPTUS
make ; make install
5. Deploying Eucalyptus
At this point, if you plan to use Eucalyptus on more thanone node, you're ready to push the software out to the other nodes (althoughnot all software components are required on all nodes, it is simpler to justmirror everything and selectively enable components via start-up scripts). Ifyou installed Eucalyptus in its own directory, you can just sync the entirepackage to all of the hosts listed above using whatever mechanism you typicallyuse to push changes to nodes (rsync, for instance).
rsync -a $EUCALYPTUS/ root@{node-host-1}:$EUCALYPTUS/
rsync -a $EUCALYPTUS/ root@{node-host-1}:$EUCALYPTUS/
...
On installations without a root user, such as Ubuntu, itmay be easier to pull the software from each node one at a time:
node1# rsync -a {user}@{front-end}:$EUCALYPTUS/ $EUCALYPTUS/
node2# rsync -a {user}@{front-end}:$EUCALYPTUS/ $EUCALYPTUS/
6. Configure Hosts
a. Set up a 'eucalyptus'user on all machines
Eucalyptus will run as regular user on your systems, whichyou'll need to add before running Eucalyptus (we will use eucalyptus) on all machines.For most distributions, this task is accomplished by running the command:
useradd eucalyptus
b. Configure yourhypervisor
Ensure that this user can control your hypervisor throughlibvirt on allcompute nodes. On some distributions, this can be accomplishedby adding eucalyptusto group libvirt or libvirtd in file /etc/group. Please consult thedocumentation for libvirt on your distribution for instructions. See HypervisorConfiguration for more detailed information.
c. Configure your network
Most networkingoptions require that, on your node controllers, the primary interface is configuredto be a bridge (this is the default configuration with some distribution's Xenhypervisor configuration). See NetworkConfiguration for more information and set-up instructions.
d. Configure Eucalyptuscomponents
On your compute nodes,create a local directory where VM images are to be placed temporarily while VMsare running (images will be cached under the same path, too). It is importantthat the directory is empty as everything in it will be removed. Be sureto pick a partition with ample disk space as VM images can be large. We use /usr/local/eucalyptus in the example below.
Place the mandatory parameters (including thetype of hypervisor you plan to use) into the configuration file and set up thepermissions on Eucalyptus files appropriately on all nodes. Both tasks can beaccomplished with flags to euca_conf tool:
- -d specifies the root of Eucalyptus installation ($EUCALYPTUS)
- --hypervisor specifies the hypervisor ('xen' or 'kvm')
- --instances specifies where, on compute nodes, instance files will be stored
- --user specifies the user that you created for running Eucalyptus
- --setup invokes the first-time setup tasks
$EUCALYPTUS/usr/sbin/euca_conf -d $EUCALYPTUS --hypervisor kvm --instances /usr/local/eucalyptus --user eucalyptus --setup
e. Distribution-specificpost configuration steps
Some linux distributions require that the admin perform afew extra steps before bringing up Eucalyptus. This section details some ofthese steps:
For Ubuntu, apparmor needs to be configured to allow dhcpd3to write to the filesystem. Add the following lines to'/etc/apparmor.d/usr.sbin.dhcp3':
/opt/eucalyptus/var/run/eucalyptus/net/ r,
/opt/eucalyptus/var/run/eucalyptus/net/** r,
/opt/eucalyptus/var/run/eucalyptus/net/*.pid lrw,
/opt/eucalyptus/var/run/eucalyptus/net/*.leases* lrw,
/opt/eucalyptus/var/run/eucalyptus/net/*.trace lrw,
where you substitute '/opt/eucalyptus' with the path towhere you have chosen to install Eucalyptus. Then, restart apparmor (NOTE:sometimes changes don't take effect right away - either wait or reboot thesystem to be sure):
/etc/init.d/apparmor stop
/etc/init.d/apparmor start
Also, since Ubuntu DHCP daemon is configured to run as'dhcpd' and not root, ensure that the following two variables are set asfollows in the $EUCALYPTUS/etc/eucalyptus/eucalyptus.conf file on the Cluster head-node:
VNET_DHCPDAEMON="/usr/sbin/dhcpd3"
VNET_DHCPUSER="dhcpd"
At this point you should be ready to start Eucalyptusprocesses on all nodes but before doing so you may want to configure theEucalyptus network: you can read more about it at NetworkConfiguration.
f. Configure your startupscripts
If you want to have eucalyptus started automatically whenyour machines are (re)booted, you can add the following symlinks on theappropriate hosts: add eucalyptus-cloud on the Cloud head-node, add eucalyptus-cc on the Cluster head-node(s), and add eucalyptus-nc on the compute node(s)
ln -sf $EUCALYPTUS/etc/init.d/eucalyptus-cloud /etc/init.d/eucalyptus-cloud
ln -sf $EUCALYPTUS/etc/init.d/eucalyptus-cc /etc/init.d/eucalyptus-cc
ln -sf $EUCALYPTUS/etc/init.d/eucalyptus-nc /etc/init.d/eucalyptus-nc
and then add the symlinks to the distribution's bootingprocess. This process differs from distribution to distribution. For example ifyou have update-rc.davailable you can run:
update-rc.d eucalyptus-cloud defaults
or if you have chkconfig available you can run:
chkconfig eucalyptus-cloud on
7. Running Eucalyptus
To run Eucalyptus,first, make sure that you have all of the runtime dependencies of Eucalyptusinstalled, based on your chosen set of configuration parameters. If there is aproblem with runtime dependencies (for instance, if Eucalyptus cannotfind/interact with them), all errors will be reported in log files located in$EUCALYPTUS/var/log/eucalyptus. For more information on Eucalyptus log filesand error reports, please see TroubleshootingEucalyptus.
Next, inspect thecontents of $EUCALYPTUS/etc/eucalyptus/eucalyptus.conf carefully, on eachmachine, to make sure that the settings are appropriate for your environment.Once you have confirmed that everything is configured properly, enable thecloud services on the front-end:
# enable services on the front-end
$EUCALYPTUS/usr/sbin/euca_conf -d $EUCALYPTUS --setup
$EUCALYPTUS/usr/sbin/euca_conf -d $EUCALYPTUS --enable cloud --enable walrus --enable sc
Ifyou are performing a first-time installation, you may proceed with thefollowing steps and start Eucalyptus services:
Start each component on the appropriate host.
# start enabled front-end services
$EUCALYPTUS/etc/init.d/eucalyptus-cloud start
# start the cluster controller
$EUCALYPTUS/etc/init.d/eucalyptus-cc start
And on each of the compute nodes run:
$EUCALYPTUS/etc/init.d/eucalyptus-nc start
To stop them you call the script with stop insteadof start.
NOTE:if you later decide to make changes to$EUCALYPTUS/etc/eucalyptus/eucalyptus.conf that will effect thecluster-controller, make sure to use the 'cleanstart', 'cleanstop', and/or'cleanrestart' directives to the init scripts (as opposed tostart/stop/restart). This will both remove all existing CC state, and willcause it to re-read the configuration file.
Source CodeStructure
The code elementscomprising Eucalyptus can be found in the following top-level directories:
- clc: This directory contains all the java code (CLC, W and SC).
- cluster: This directory contains the CC code only.
- gatherlog: This directory contains the GL code. It is a simple C web service used to access eucalyptus logs on remote machines. Given its simplicity, this is a good starting point for familiarizing yourself with axis2c (the framework used to write the C web service within eucalyptus) and rampartc (implementation of WS-Security for axis2c);
- net: Contains the code used by the CC and the NC to setup the virtual network for the instances.
- node: Contains the NC. The NC uses libvirt to interact with the hypervisor, though some hypervisor specific code is still needed (available in handlers_.c).
- storage: Contains the code used by the NC to interact with W (to get the images).
- tools: Contains the command line tools used in eucalyptus (for example euca_conf), the default configuration file, scripts used when starting the instances or interacting with the hypervisor, and finally certain scripts used in the build process.
- util: Contains common functions used by the CC and NC.
- wsdl: Contains the current wsdl for the CC, NC and GL.
- debian: This directory will be going away, since we'll have it in a different repository, making it easier to maintain the debian scripts.
- extras: Contains scripts which may be used with third party tools (for example to interact with nagios and ganglia).
Debugging
Once you begin modifying Eucalyptus code, you will findthat debugging your changes is not trivial—due to the distributed nature ofEucalyptus and the use of WS-Security. Here we provide some tips to help youdebug your code. In the following, we assume that you install eucalyptus in/opt/eucalytpus.
- make sure the shared memory is cleaned:
ls /dev/shm rm /dev/shm/euca* - running the NC ina debugger:
First, make sure that the the NCis stopped. You can run the debug the NC either on the web server provided byaxis2c or in apache. For the axis2c web server you can:
export EUCALYPTUS=/opt/eucalyptus/
export AXIS2C_HOME=/opt/eucalyptus/packages/axis2c-1.6.0/
gdb $AXIS2C_HOME/bin/axis2_http_servergdb> run -p 8775 -r $AXIS2C_HOME-f axis2_httpd_server_error.log -l 2
while for apache you can
export EUCALYPTUS=/opt/eucalyptus/
export AXIS2C_HOME=/opt/eucalyptus/packages/axis2c-1.6.0/
export LD_LIBRARY_PATH=$AXIS2C_HOME/lib:/opt/eucalyptus/packages/axis2c-1.6.0/modules/rampart/
gdb /usr/sbin/apache2 run -X -f/opt/eucalyptus/etc/eucalyptus/httpd-nc.conf
Check the wire: to check what's been sent on the wirebetween component, you can use ngrep and the port of interest (in the followingexample the CLC port): sudongrep -d INTERFACE -P' ' -Wbyline port 8773 or sudo tcpdump -i INTERFACE -n -s0 -A port 877
- To start a Java process in debug mode:
- Run eucalyptus-cloud with --debug to have it listen on port 5005.
- Add the flag to CLOUD_OPTS in eucalyptus.conf
- To have the jvm stop at 'main()' also add the flag '--debug-suspend'.
DebuggingGWT-based Eucalyptus code in Eclipse
Client-side Web UIcode can be debugged in Eclipse with a stubbed out server.
- Prerequisites: in Eclipse
- File → New → Java Project
- Choose “Create project from existing source”
- Browse for clc/modules/www/test-gwt
- Finish
- Build and run:
- Build the complete eucalyptus project ONCE.
- Add external jars to the test-gwt project:
- clc/target/eucalyptus-auth-main.jar
- clc/target/eucalyptus-msgs-main.jar
- clc/lib/groovy-1.6.3.jar
- Make sure there is no problem in the Problems tab.
- Right-click on project → Google → GWT Compile (for hosted/ development mode this needs to happen only once if you have Project → Build Automatically chosen)
- Right-click on project → Debug As → Web Application
- To run in hosted/development mode, go to http://localhost:8888/edu.ucsb.eucalyptus.admin.EucalyptusWebInterface/?gwt.code svr=127.0.0.1:9997
- TODO VERIFY: To run in full mode, go to http://localhost:8888/edu.ucsb.eucalyptus.admin.EucalyptusWebInterface/
