[转]Part1 – OPENVSWICH – Creating and Submitting Openvswitch Extension To Tinycore Upstream

In May 2011, I read a request for installation Openvswitch on Qemu image. I started to play with Openvswitch and finally became fan of this project. I realized how powerful can be Openvswitch, offering many features listed here.  Spending my time playing with Microcore and Openvswitch after several days I was able to create Microcore Qemu image with 8021q support  and install Openvswitch on the top of it.

In the tutorial I would like to show how to create Openvswitch extension and make it ready for submit to the Tinycore upstream.  I chose Core Linux, because I am familiar with this minimal Linux distribution and the Core is incredibly small.

The following steps describe installation Openvswitch on Qemu image with pre-installed Microcore Linux.

I also created three labs using Openvswitch. I tested how can Openvswitch works with VLANs, 802.1q trunk ports,  if it was capable of creating L3 VLAN interfaces and Inter VLAN routing was working.  Labs are available here:

1. Start Qemu Microcore image

Assuming your Microcore Qemu image with console support has been created and supports 8021q VLAN tagging, start the image:

qemu-kvm /home/brezular/linux-microcore-4.7.7.img -m 512

2. Download Openvswitch and Extract it

wget http://openvswitch.org/releases/openvswitch-1.11.0.tar.gz

tar zxvf ./openvswitch-1.11.0.tar.gz

cd ./openvswitch-1.1.0

3. Install Necessary Dependences for Openvswitch Compilation

tce-load -w -i compiletc.tcz python.tcz perl5.tcz openssl-1.0.0-dev.tcz tunctl.tcz bridge-utils.tcz linux-headers-3.0.21-tinycore.tcz

To compile openvswitch with  the ovsdmonitor tool, the machine must also have the following installed software:

tce-load -w -i python_twisted-2.7.tcz python-simplejson.tcz python_zope.interface-2.7 pyqt.tcz

Note:  The kernel source will be saved in /lib/modules/`uname -r`/build

It is recommended to check the list of necessary applications here:

4. Openvswitch Installation

It is recommended to start here and continue here :-) After that you can type following:

export CFLAGS="-march=i486 -mtune=i686 -Os -pipe"
export CXXFLAGS="-march=i486 -mtune=i686 -Os -pipe"
export LDFLAGS="-Wl,-O1"

cd ./openvswitch-1.11.0

./configure --prefix=/usr/local --with-linux=/lib/modules/`uname -r`/build

make -j2

sudo su

cd /home/tc/openvswitch-1.11.0/

make  DESTDIR=/tmp/openvswitch install

5. BACKUP and LOAD Module openvswitch.ko

a) BACKUP module openvswitch.ko

After building, a kernel module “openvswitch.ko” will be saved in a ./datapath/linux/ directory. We’ve built both userspace and kernel module as well.  The performance is better when a kernel module is used. Create a directory in a structure of the openvswitch extension where the kernel module openvswitch.ko will be saved.

sudo mkdir -p /tmp/openvswitch/usr/local/lib/modules/3.0.21-tinycore/kernel/openvswitch/

Copy the kernel module to the directory.

sudo cp /home/tc/openvswitch-1.11.0/datapath/linux/openvswitch.ko  /tmp/openvswitch/usr/local/lib/modules/3.0.21-tinycore/kernel/openvswitch/

b) Check if module can be loaded correctly

sudo mkdir -p /usr/local/lib/modules/3.0.21-tinycore/kernel/openvswitch/

sudo cp /home/tc/openvswitch-1.11.0/datapath/linux/openvswitch.ko  /usr/local/lib/modules/3.0.21-tinycore/kernel/openvswitch/

depmod -a

sudo modprobe openvswitch

Check if module openvswitch_mod.ko is loaded to the kernel with lsmod command.

lsmod | grep openvswitch
openvswitch 49152 0

6. Backup /home/tc/openvswitch-1.11.0/vswitchd/vswitch.ovsschema

In order to initialize the configuration database using ovsdb-tool later, the file /home/tc/openvswitch-1.11.0/vswitchd/vswitch.ovsschema is needed. You need to copy it to /tmp/openvswitch/usr/local/etc/openvswitch/vswitchd/ to become part of the extension.

sudo mkdir -p /tmp/openvswitch/usr/local/etc/openvswitch/vswitchd/

sudo cp /home/tc/openvswitch-1.11.0/vswitchd/vswitch.ovsschema /tmp/openvswitch/usr/local/etc/openvswitch/vswitchd/

7. Create Openvswitch Extension

a) Remove unneccessary files

sudo rm -rf /tmp/openvswitch/usr/local/share/man/

b) Install squashfs and create openvswitch-3.0.21-tinycore.tcz extension

tce-load -w -i squashfs-tools-4.x

sudo su
cd /tmp/
mksquashfs openvswitch/ openvswitch-3.0.21-tinycore.tcz

c) Create a list of files presented in extension

sudo su
cd /tmp/openvswitch/
find usr -not -type d > ../openvswitch-3.0.21-tinycore.tcz.list
cd ..

d) Create md5 check sum of openvswitch-3.0.21-tinycore.tcz

md5sum openvswitch-3.0.21-tinycore.tcz > openvswitch-3.0.21-tinycore.tcz.md5.txt

e) Create openvswitch-3.0.21-tinycore.tcz.info

An info file describing its contents (.tcz.info) – this content is standardized. Check repository for examples.

f) Create openvswitch-3.0.21-tinycore.tcz.build-dep file

Additional build instructions in a plain text file for future reference, mentioning such things as which extensions are required to build the package and what compile flags were used.

g) Create the dependency list openvswitch-3.0.21-tinycore.tcz.dep

List of the extensions that have to be presented to run openvswitch extension correctly.

h) Install  openssh.tcz an extension and copy all openvswitch-3.0.21-tinycore.tcz.*  files and a source file to the guest system

tce-load -wi openssh

cp -rv /home/tc/openvswitch-1.11.0.tar.gz /tmp

scp -rv /tmp/openvswitch-3.0.21-tinycore.* brezular@10.0.2.2:/home/brezular/
scp -rv /tmp/openvswitch-1.11.0.tar.gz brezular@10.0.2.2:/home/brezular/

8. Test and Submit Openvswitch Extension

a) Create the new  clean Microcore Qemu image without any extensions installed

Assuming the image is ready to start, run it:

qemu-kvm /home/brezular/linux-microcore-4.7.7-clean.img -m 512

b) Install submitqc.tcz and openssh.tcz

tce-load -wi submitqc4.tcz openssh.tcz

c) Copy the openvswitch files you have created in steps 1 – 7

(.tcz, .list, .md5.txt, .info, .dep, build-dep) from old Qemu image to the clean Qemu image, to the directory  /tmp/All.

mkdir /tmp/All
cd /tmp/All

scp -rv brezular@10.0.2.2:/home/brezular/linux-core-nemaz/extensions/openvswitch/* .

Run an extension testing script:

sudo submitqc4

The script checks all openvswitch files in /tmp/All and creates the directory /tmp/submitqc/ with log files.

d) Send extension

sudo mv /tmp/submitqc /tmp/All
cd /tmp/All

sudo tar zcvf openvswitch-1.11.0-extension.tar.gz *

Send openvswitch-1.11.0-extension.tar.gz  to  tcesubmit@gmail.com

e) Copy new created files to the guest

sudo scp -rv submitqc/ brezular@10.0.2.2:/home/brezular/
sudo scp -rv openvswitch-3.0.21-tinycore.tcz.zsync brezular@10.0.2.2:/home/brezular/
sudo scp -rv openvswitch-1.11.0-extension.tar.gz brezular@10.0.2.2:/home/brezular/

9. Openvswitch After-Install Configuration

a) Make openvswitch, 8021q, ipv6 modules to be loaded to a kernel during boot of Microcore

echo "modprobe openvswitch" >> /opt/bootlocal.sh
echo "modprobe 8021q" >> /opt/bootlocal.sh
echo "modprobe ipv6" >> /opt/bootlocal.sh

sudo modprobe openvswitch_mod
sudo modprobe 8021q
sudo modprobe ipv6

b) Initialize the configuration database using ovsdb-tool

Check if a directory /usr/local/etc/openvswitch/ exists, if not create it.

sudo mkdir -p /usr/local/etc/openvswitch/

Create conf.db configuration file.

sudo ovsdb-tool create /usr/local/etc/openvswitch/conf.db /usr/local/etc/openvswitch/vswitchd/vswitch.ovsschema

Add /usr/local/etc/openvswitch/ to the list of after restart kept files.

echo "/usr/local/etc/openvswitch/" >> /opt/.filetool.lst

c) Make ovsdb-server to be started after start of the Microcore

vi /opt/bootlocal.sh

/usr/local/sbin/ovsdb-server --remote=punix:/usr/local/var/run/openvswitch/db.sock 
                             --remote=db:Open_vSwitch,Open_vSwitch,manager_options 
                             --private-key=db:Open_vSwitch,SSL,private_key 
                             --certificate=db:Open_vSwitch,SSL,certificate 
                             --bootstrap-ca-cert=db:Open_vSwitch,SSL,ca_cert 
                             --pidfile --detach

:wq!

Note:  If you are not familiar with vi editor, use the reference below, please:

http://nemesis.lonestar.org/reference/docs/vi.html

d) Make the database initialialization using ovs-vsctl

This is only necessary the first time after you create the database with ovsdb-tool (but running it at any time is harmless).

echo "/usr/local/bin/ovs-vsctl --no-wait init" >> /opt/bootlocal.sh

e) Make the main Open vSwitch daemon being started, telling it to connect to the same Unix domain socket

echo "/usr/local/sbin/ovs-vswitchd --pidfile --detach" >> /opt/bootlocal.sh

f) Enable IPv4 and IPV6 packets forwarding between interfaces

Although not directly connected with Openvswitch configuration we need to enable ipv4 and ipv6 packets forwarding between interfaces for Microcore.  This option  is disabled in kernel by default.

sudo sysctl -w net.ipv4.ip_forward=1
sudo sysctl -w net.ipv6.conf.all.forwarding=1

echo "sysctl -w net.ipv4.ip_forward=1" >> /opt/bootlocal.sh
echo "sysctl -w net.ipv6.conf.all.forwarding=1" >> /opt/bootlocal.sh

g) Run commands you have entered into  /opt/bootlocal.sh and make them persistent after restart of the Microcore

sudo /opt/bootlocal.sh

Optionally: It is recommended to delete the history of used commands.

echo "" > /home/tc/.ash_history

Save changes made for files and directories which are listed in  /opt/.filetool.lst

/usr/bin/filetool.sh -b

10. Configuration example

Now you may use ovs-vsctl to set up bridges and other Open vSwitch features.  For example, to create a bridge named br0 and add ports eth0, eth1  and eth2 to it:

sudo ovs-vsctl add-br br0

sudo ovs-vsctl add-port br0 eth0
sudo ovs-vsctl add-port br0 eth1
sudo ovs-vsctl add-port br0 eth2

Before shutdown you always force Core to save configuration changes in the openvswitch database file –  /usr/local/etc/openvswitch/conf.db. Use the command:

/usr/bin/filetool.sh -b

Reference
http://openvswitch.org/
http://openvswitch.org/cgi-bin/gitweb.cgi?p=openvswitch;a=blob_plain;f=INSTALL.userspace;hb=HEAD
http://openvswitch.org/?page_id=14
http://openvswitch.org/cgi-bin/gitweb.cgi?p=openvswitch;a=blob_plain;f=INSTALL.Linux;hb=HEAD