Deploying Cloud Foundry on OpenStack Juno and XenServer (Part II)
link
http://rabbitstack.github.io/deploying-cloud-foundry-on-openstack-juno-and-xenserver-part-ii/
Let's move on. We should have our OpenStack instance prepared for Cloud Foundry. The most usual way of deploying Cloud Foundry is through BOSH. For the who still didn't hear about it, BOSH is the platform for automation and lifecycle management of software and distributed services. It is also capable of monitoring and failure recovery of processes and virtual machines. There are already a few IT automation platforms in the market like Chef or Puppet, so, why to learn / use BOSH then?
One notable difference is that BOSH is able to perform the deployment from the sterile environment, i.e. package source code and dependencies, create the virtual machines (jobs in BOSH terminology) from the so calledstemcell template (VM which has BOSH agent installed and is used to generate the jobs), and finally install, start and monitor the required services and VMs. Visit the official page from the link above to learn more about BOSH.
Deploying MicroBOSH
MicroBOSH is a single VM which contains all the necessary components to boot BOSH, including the blobstore, nats, director, health manager etc. Once you have an instance of MicroBOSH running, you can deploy BOSH if you wish. Install BOSH CLI gems (Ruby >= 1.9.3 is required).
$ gem install bosh_cli bosh_cli_plugin_micro
You will need to create a keypair in OpenStack and configure bosh
security group with the rules shown in the table below. You can do it by accessing the Horizon dashboard or by using nova
CLI.
Direction | IP Protocol | Port Range | Remote |
---|---|---|---|
Ingress | TCP | 1-65535 | bosh |
Ingress | TCP | 53 (DNS) | 0.0.0.0/0 (CIDR) |
Ingress | TCP | 4222 | 0.0.0.0/0 (CIDR) |
Ingress | TCP | 6868 | 0.0.0.0/0 (CIDR) |
Ingress | TCP | 4222 | 0.0.0.0/0 (CIDR) |
Ingress | TCP | 25250 | 0.0.0.0/0 (CIDR) |
Ingress | TCP | 25555 | 0.0.0.0/0 (CIDR) |
Ingress | TCP | 25777 | 0.0.0.0/0 (CIDR) |
Ingress | UDP | 53 | 0.0.0.0/0 (CIDR) |
Ingress | UDP | 68 | 0.0.0.0/0 (CIDR) |
$ nova keypair-add microbosh > microbosh.pem
$ chmod 600 microbosh.pem
BOSH uses a variety of artifacts in order to complete the deployment life cycle. We can basically distinguish between stemcell, release and deployment. To deploy MicroBOSH we will only need a stemcell which can be downloaded using the bosh CLI. First get a list of available stemcells and download thebosh-stemcell-2839-openstack-kvm-centos-go_agent-raw.tgz
.
$ bosh public stemcells
+-----------------------------------------------------------------+
| Name |
+-----------------------------------------------------------------+
| bosh-stemcell-2427-aws-xen-ubuntu.tgz |
| bosh-stemcell-2652-aws-xen-centos.tgz |
| bosh-stemcell-2839-aws-xen-centos-go_agent.tgz |
| bosh-stemcell-2427-aws-xen-ubuntu-go_agent.tgz |
| bosh-stemcell-2710-aws-xen-ubuntu-lucid-go_agent.tgz |
| bosh-stemcell-2652-aws-xen-ubuntu-lucid.tgz |
| bosh-stemcell-2839-aws-xen-ubuntu-trusty-go_agent.tgz |
| bosh-stemcell-2690.6-aws-xen-ubuntu-trusty-go_agent.tgz |
| bosh-stemcell-2719.1-aws-xen-centos-go_agent.tgz |
| bosh-stemcell-2719.1-aws-xen-ubuntu-trusty-go_agent.tgz |
| bosh-stemcell-2719.2-aws-xen-centos-go_agent.tgz |
| bosh-stemcell-2719.2-aws-xen-ubuntu-trusty-go_agent.tgz |
| bosh-stemcell-2719.3-aws-xen-ubuntu-trusty-go_agent.tgz |
| light-bosh-stemcell-2427-aws-xen-ubuntu.tgz |
| light-bosh-stemcell-2652-aws-xen-centos.tgz |
| light-bosh-stemcell-2839-aws-xen-centos-go_agent.tgz |
| light-bosh-stemcell-2427-aws-xen-ubuntu-go_agent.tgz |
| light-bosh-stemcell-2710-aws-xen-ubuntu-lucid-go_agent.tgz |
| light-bosh-stemcell-2652-aws-xen-ubuntu-lucid.tgz |
| light-bosh-stemcell-2839-aws-xen-ubuntu-trusty-go_agent.tgz |
| light-bosh-stemcell-2690.6-aws-xen-ubuntu-trusty-go_agent.tgz |
| light-bosh-stemcell-2719.1-aws-xen-centos-go_agent.tgz |
| light-bosh-stemcell-2719.1-aws-xen-ubuntu-trusty-go_agent.tgz |
| light-bosh-stemcell-2719.2-aws-xen-centos-go_agent.tgz |
| light-bosh-stemcell-2719.2-aws-xen-ubuntu-trusty-go_agent.tgz |
| light-bosh-stemcell-2719.3-aws-xen-ubuntu-trusty-go_agent.tgz |
| light-bosh-stemcell-2839-aws-xen-hvm-centos-go_agent.tgz |
| light-bosh-stemcell-2839-aws-xen-hvm-ubuntu-trusty-go_agent.tgz |
| bosh-stemcell-2427-openstack-kvm-ubuntu.tgz |
| bosh-stemcell-2624-openstack-kvm-centos.tgz |
| bosh-stemcell-2624-openstack-kvm-ubuntu-lucid.tgz |
| bosh-stemcell-2839-openstack-kvm-centos-go_agent.tgz |
| bosh-stemcell-2839-openstack-kvm-ubuntu-trusty-go_agent.tgz |
| bosh-stemcell-2652-openstack-kvm-ubuntu-lucid-go_agent.tgz |
| bosh-stemcell-2719.1-openstack-kvm-centos-go_agent.tgz |
| bosh-stemcell-2719.1-openstack-kvm-ubuntu-trusty-go_agent.tgz |
| bosh-stemcell-2719.2-openstack-kvm-centos-go_agent.tgz |
| bosh-stemcell-2719.2-openstack-kvm-ubuntu-trusty-go_agent.tgz |
| bosh-stemcell-2719.3-openstack-kvm-ubuntu-trusty-go_agent.tgz |
| bosh-stemcell-2839-openstack-kvm-centos-go_agent-raw.tgz |
| bosh-stemcell-2839-openstack-kvm-ubuntu-trusty-go_agent-raw.tgz |
| bosh-stemcell-2427-vcloud-esxi-ubuntu.tgz |
| bosh-stemcell-2652-vcloud-esxi-ubuntu-lucid.tgz |
| bosh-stemcell-2839-vcloud-esxi-ubuntu-trusty-go_agent.tgz |
| bosh-stemcell-2690.5-vcloud-esxi-ubuntu-trusty-go_agent.tgz |
| bosh-stemcell-2690.6-vcloud-esxi-ubuntu-trusty-go_agent.tgz |
| bosh-stemcell-2710-vcloud-esxi-ubuntu-lucid-go_agent.tgz |
| bosh-stemcell-2427-vsphere-esxi-ubuntu.tgz |
| bosh-stemcell-2624-vsphere-esxi-centos.tgz |
| bosh-stemcell-2839-vsphere-esxi-centos-go_agent.tgz |
| bosh-stemcell-2427-vsphere-esxi-ubuntu-go_agent.tgz |
| bosh-stemcell-2710-vsphere-esxi-ubuntu-lucid-go_agent.tgz |
| bosh-stemcell-2624-vsphere-esxi-ubuntu-lucid.tgz |
| bosh-stemcell-2839-vsphere-esxi-ubuntu-trusty-go_agent.tgz |
| bosh-stemcell-2719.1-vsphere-esxi-centos-go_agent.tgz |
| bosh-stemcell-2719.1-vsphere-esxi-ubuntu-trusty-go_agent.tgz |
| bosh-stemcell-2719.2-vsphere-esxi-ubuntu-trusty-go_agent.tgz |
| bosh-stemcell-2719.2-vsphere-esxi-centos-go_agent.tgz |
| bosh-stemcell-2719.3-vsphere-esxi-ubuntu-trusty-go_agent.tgz |
| bosh-stemcell-2690.6-vsphere-esxi-ubuntu-trusty-go_agent.tgz |
| bosh-stemcell-389-warden-boshlite-ubuntu-trusty-go_agent.tgz |
| bosh-stemcell-53-warden-boshlite-ubuntu.tgz |
| bosh-stemcell-389-warden-boshlite-centos-go_agent.tgz |
| bosh-stemcell-64-warden-boshlite-ubuntu-lucid-go_agent.tgz |
+-----------------------------------------------------------------+
$ bosh download public stemcell bosh-stemcell-2839-openstack-kvm-centos-go_agent-raw.tgz
bosh-stemcell: 4% |ooo | 24.4MB 753.0KB/s ETA: 00:11:43
Now we are ready to create the MicroBOSH deployment manifestmicrobosh-openstack.yml
file. You will need to change net_id
with your OpenStack instance network identifier, ip
with the ip address from the network pool. You can find out that information by executing the following commands.
$ nova network-list
+--------------------------------------+----------+----------------+
| ID | Label | Cidr |
+--------------------------------------+----------+----------------+
| 3f36d40e-1097-49a0-a023-4606dbf3a1f5 | yuna-net | 192.168.1.0/24 |
+--------------------------------------+----------+----------------+
$ nova network-show 3f36d40e-1097-49a0-a023-4606dbf3a1f5
+---------------------+--------------------------------------+
| Property | Value |
+---------------------+--------------------------------------+
| bridge | xenbr0 |
| bridge_interface | eth0 |
| broadcast | 192.168.1.255 |
| cidr | 192.168.1.0/24 |
| cidr_v6 | - |
| created_at | 2014-12-28T17:18:14.000000 |
| deleted | False |
| deleted_at | - |
| dhcp_server | 192.168.1.50 |
| dhcp_start | 192.168.1.51 |
| dns1 | 8.8.4.4 |
| dns2 | - |
| enable_dhcp | True |
| gateway | 192.168.1.50 |
| gateway_v6 | - |
| host | - |
| id | 3f36d40e-1097-49a0-a023-4606dbf3a1f5 |
| injected | False |
| label | yuna-net |
| mtu | - |
| multi_host | True |
| netmask | 255.255.255.0 |
| netmask_v6 | - |
| priority | - |
| project_id | - |
| rxtx_base | - |
| share_address | True |
| updated_at | - |
| vlan | - |
| vpn_private_address | - |
| vpn_public_address | - |
| vpn_public_port | - |
+---------------------+--------------------------------------+
Under the openstack
section change the Identity service endpoint, OpenStack credentials, the private key location, and optionally set the timeout for OpenStack resources.
---
name: microbosh-openstack
logging:
level: DEBUG
network:
type: manual
ip: 192.168.1.55
cloud_properties:
net_id: 3f36d40e-1097-49a0-a023-4606dbf3a1f5
resources:
persistent_disk: 16384
cloud_properties:
instance_type: m1.medium
cloud:
plugin: openstack
properties:
openstack:
auth_url: http://controller:5000/v2.0
username: admin
api_key: admin
tenant: admin
default_security_groups: ["bosh"]
default_key_name: microbosh
private_key: /root/microbosh.pem
state_timeout: 900
apply_spec:
properties:
director:
max_threads: 3
hm:
resurrector_enabled: true
ntp:
- 0.europe.pool.ntp.org
- 1.europe.pool.ntp.org
Finally, set the current deployment manifest file and deploy MicroBOSH.
$ bosh micro deployment microbosh-openstack.yml
$ bosh micro deploy bosh-stemcell-2839-openstack-kvm-centos-go_agent-raw.tgz
If everything goes well you should login into the MicroBOSH instance (use admin
, for both username and password).
$ bosh target 192.168.1.55
Target set to 'microbosh-openstack'
Your username: admin
Enter password: *****
Logged in as 'admin'
Deploying Cloud Foundry
Start by cloning the Cloud Foundry repository. Enter the newly created cf-release
directory and execute the update
script to update all submodules.
$ git clone https://github.com/cloudfoundry/cf-release.git
$ cd cf-release
$ ./update
Upload the stemcell to the BOSH Director.
$ bosh upload stemcell bosh-stemcell-2839-openstack-kvm-centos-go_agent-raw.tgz
In BOSH terminology, release is a collection of packages and source code, dependencies, configuration properties, and any other components required to perform a deployment. To create a Cloud Foundry release, use this command fromcf-release
directory.
$ bosh create release
This will download the required blobs from the S3 storage service and generate a release tarball. You should end up with the similar directory structures.
$ ls blobs
buildpack_cache git haproxy mysql php-buildpack rootfs ruby-buildpack
cli go-buildpack java-buildpack nginx postgres ruby sqlite
debian_nfs_server golang libyaml nodejs-buildpack python-buildpack ruby-2.1.4 uaa
$ ls packages
acceptance-tests buildpack_python dea_next golang loggregator_trafficcontroller postgres warden
buildpack_cache buildpack_ruby debian_nfs_server golang1.3 login rootfs_lucid64
buildpack_go cli doppler gorouter metron_agent ruby
buildpack_java cloud_controller_ng etcd haproxy mysqlclient ruby-2.1.4
buildpack_java_offline collector etcd_metrics_server hm9000 nats smoke-tests
buildpack_nodejs common git libpq nginx sqlite
buildpack_php dea_logging_agent gnatsd libyaml nginx_newrelic_plugin uaa
Now you can upload the release to the BOSH Director.
$ bosh upload release
The most complex part of Cloud Foundry BOSH deployment is the manifest file where all components are tied together - computing resource specifications, VMs, software releases, and configuration properties. You can use the deployment which worked great on my environment. Don’t forget to create cf.small
and cf.medium
flavors in OpenStack.
123456789101112131415161718192021222324252627282930313233343536373839404142434445464748495051525354555657585960616263646566676869707172737475767778798081828384858687888990919293949596979899100101102103104105106107108109110111112113114115116117118119120121122123124125126127128129130131132133134135136137138139140141142143144145146147148149150151152153154155156157158159160161162163164165166167168169170171172173174175176177178179180181182183184185186187188189190191192193194195196197198199200201202203204205206207208209210211212213214215216217218219220221222223224225226227228229230231232233234235236237238239240241242243 |