A layer two VPN tunnel connects broadcast domains, not subnets. Also, tunneling the same subnet in different namespaces does not magically convert the layer three VPN to a layer two VPN.
1, Use devstack to install a all-in-one openstack env with strongswan driver in 5 minutes.
NOTE:
a, Only supports ubuntu 14.04 now, devstack will help us disable apparmor in 14.04 by the following commands:
sudo ln -sf /etc/apparmor.d/usr.lib.ipsec.charon /etc/apparmor.d/disable/
sudo ln -sf /etc/apparmor.d/usr.lib.ipsec.stroke /etc/apparmor.d/disable/
# NOTE: Due to https://bugs.launchpad.net/ubuntu/+source/apparmor/+bug/1387220
# one must use 'sudo start apparmor ACTION=reload' for Ubuntu 14.10
sudo service apparmor restart
b, No any network related requirements for your node. devstack will create br-phy and br-ex automatically.
Both virtual machine and physical machine are ok, any network configurations are ok.
But need to be able to access to the Internet.
c, Assume openstack work directory is: /bak/openstack, make sure you are using a non-root user like hua,
then change the file permissions for /bak/openstack directory ( sudo chown -R hua:root /bak/openstack ).
The next steps are below:
a, Download devstack
cd /bak/openstack && git clone https://github.com/openstack-dev/devstack.git
b, Paste the following content to the file /bak/openstack/devstack/localrc ,
then run devstack by the command:
cd /bak/openstack/devstack && ./stack.sh
# Use strongswan driver code, https://review.openstack.org/#/c/144391/
DEST=/bak/openstack
IPSEC_PACKAGE=strongswan
sudo route del -net 10.0.1.0/24 gw 192.168.101.3
sudo apt-get install openvswitch-switch qemu-kvm libvirt-bin
sudo ovs-vsctl -- --may-exist add-br br-phy
sudo ifconfig br-phy 172.16.1.1/24
ENABLED_SERVICES=rabbit,mysql,key,g-api,g-reg,tempest
ENABLED_SERVICES+=,n-api,n-crt,n-obj,n-cpu,n-cond,n-sch
ENABLED_SERVICES+=,cinder,c-api,c-vol,c-sch
ENABLED_SERVICES+=,q-svc,q-agt,q-dhcp,q-l3,q-meta,neutron,q-lbaas,q-fwaas,q-vpn
HOST_IP=172.16.1.1
SERVICE_HOST=$HOST_IP
MYSQL_HOST=$SERVICE_HOST
RABBIT_HOST=$SERVICE_HOST
GLANCE_HOSTPORT=$SERVICE_HOST:9292
Q_HOST=$SERVICE_HOST
FIXED_RANGE=10.0.1.0/24
FLOATING_RANGE=192.168.101.0/24
Q_FLOATING_ALLOCATION_POOL=start=192.168.101.3,end=192.168.101.100
PUBLIC_NETWORK_GATEWAY=192.168.101.1
NETWORK_GATEWAY=10.0.1.1
PUBLIC_BRIDGE=br-ex
# sudo ovs-vsctl add-port br-ex eth0
OVS_PHYSICAL_BRIDGE=br-phy
DATABASE_USER=root
DATABASE_PASSWORD=password
ADMIN_PASSWORD=password
SERVICE_PASSWORD=password
RABBIT_PASSWORD=password
SERVICE_TOKEN=ADMIN
LOGFILE=$DEST/logs/stack.log
ENABLE_DEBUG_LOG_LEVEL=False
SYSLOG=False
SCREEN_LOGDIR=$DEST/logs
LOG_COLOR=False
Q_USE_DEBUG_COMMAND=False
APACHE_ENABLED_SERVICES+=keystone
KEYSTONE_TOKEN_FORMAT=uuid
USE_SSL=False
disable_service tls-proxy
IMAGE_URLS+=",http://cdn.download.cirros-cloud.net/0.3.2/cirros-0.3.2-x86_64-disk.img"
2, I often use vpnaas tempest scenario test (https://review.openstack.org/#/c/153292/) to speed up to set up vpn env.
But this time I will demo how to use CLI commands to set up vpn tunnel between two routers with two namespaces in a all-in-one env.
(10.0.2.0/24 - DevStack East)
|
| 10.0.2.1
[Neutron Router]
| 192.168.101.4
|
| 192.168.101.1
[Internet GW]
|
|
[Internet GW]
| 192.168.101.1
|
| 192.168.101.5
[Nuetron Router]
| 10.0.3.1
|
(10.0.3.0/24 DevStack West)
a, Create network elements On East and West
export OS_USERNAME=admin
export OS_PASSWORD=password
export OS_TENANT_NAME=demo
export OS_AUTH_URL=http://172.16.1.1:5000/v2.0
export OS_AUTH_STRATEGY=keystone
TENANT_ID=$(keystone tenant-list |grep ' admin ' |awk '{print $2}')
EXT_NET_ID=$(neutron net-list |grep ' public ' |awk '{print $2}')
neutron net-create vpn-net-east --provider:network_type vxlan --provider:segmentation_id 1012
neutron subnet-create --tenant_id $TENANT_ID --ip_version 4 --gateway 10.0.2.1 vpn-net-east 10.0.2.0/24
neutron router-create --tenant_id $TENANT_ID vpn-router-east
EAST_ROUTER_ID=$(neutron router-list |grep ' vpn-router-east ' |awk '{print $2}')
EAST_SUBNET_ID=$(neutron subnet-list |grep '10.0.2.0/24' |awk '{print $2}')
neutron router-interface-add $EAST_ROUTER_ID $EAST_SUBNET_ID
# devstack has helped us create public network
# neutron net-create public -- --router:external=True
# neutron subnet-create --allocation-pool start=192.168.101.3,end=192.168.101.100 --gateway 192.168.101.1 public 192.168.101.0/24 --enable_dhcp=False
neutron router-gateway-set $EAST_ROUTER_ID $EXT_NET_ID
TENANT_ID=$(keystone tenant-list |grep ' admin ' |awk '{print $2}')
EXT_NET_ID=$(neutron net-list |grep ' public ' |awk '{print $2}')
neutron net-create vpn-net-west --provider:network_type vxlan --provider:segmentation_id 1013
neutron subnet-create --tenant_id $TENANT_ID --ip_version 4 --gateway 10.0.3.1 vpn-net-west 10.0.3.0/24
neutron router-create --tenant_id $TENANT_ID vpn-router-west
WEST_ROUTER_ID=$(neutron router-list |grep ' vpn-router-west ' |awk '{print $2}')
WEST_SUBNET_ID=$(neutron subnet-list |grep '10.0.3.0/24' |awk '{print $2}')
neutron router-interface-add $WEST_ROUTER_ID $WEST_SUBNET_ID
neutron router-gateway-set $WEST_ROUTER_ID $EXT_NET_ID
b, Create two VMs On East and West
nova keypair-add --pub_key ~/.ssh/id_rsa.pub mykey
IMAGE_ID=$(nova image-list |grep 'cirros' |awk '{print $2}')
nova secgroup-add-rule default tcp 22 22 0.0.0.0/0
nova secgroup-add-rule default icmp -1 -1 0.0.0.0/0
EAST_NET_ID=$(neutron net-list |grep ' vpn-net-east ' |awk '{print $2}')
nova boot --key_name mykey --image $IMAGE_ID --flavor 1 --nic net-id=$EAST_NET_ID vpn-vm-east
EAST_VM_ID=$(nova list |grep 'vpn-vm-east' |awk '{print $2}')
FLOATING_IP_EAST=$(nova floating-ip-create |grep ' public ' |awk '{print $4}')
nova add-floating-ip $EAST_VM_ID $FLOATING_IP_EAST
ssh -i mykey.priv cirros@$FLOATING_IP_EAST
WEST_NET_ID=$(neutron net-list |grep ' vpn-net-west ' |awk '{print $2}')
nova boot --key_name mykey --image $IMAGE_ID --flavor 1 --nic net-id=$WEST_NET_ID vpn-vm-west
WEST_VM_ID=$(nova list |grep 'vpn-vm-west' |awk '{print $2}')
FLOATING_IP_WEST=$(nova floating-ip-create |grep 'public' |awk '{print $4}')
nova add-floating-ip $WEST_VM_ID $FLOATING_IP_WEST
ssh -i mykey.priv cirros@$FLOATING_IP_WEST
c, Create VPN elements On East and West
EAST_EXT_IP=$(neutron router-show vpn-router-east |grep 'external_gateway_info' |grep -o '[0-9]\{1,3\}\.[0-9]\{1,3\}\.[0-9]\{1,3\}\.[0-9]\{1,3\}')
WEST_EXT_IP=$(neutron router-show vpn-router-west |grep 'external_gateway_info' |grep -o '[0-9]\{1,3\}\.[0-9]\{1,3\}\.[0-9]\{1,3\}\.[0-9]\{1,3\}')
neutron vpn-ikepolicy-create ikepolicy1
neutron vpn-ipsecpolicy-create ipsecpolicy1
neutron vpn-service-create --name vpn-east --description "VPN EAST" $EAST_ROUTER_ID $EAST_SUBNET_ID
neutron ipsec-site-connection-create --name vpn_conn_east --vpnservice-id vpn-east --ikepolicy-id ikepolicy1 --ipsecpolicy-id ipsecpolicy1 --peer-address $WEST_EXT_IP --peer-id $WEST_EXT_IP --peer-cidr 10.0.3.0/24 --psk password
neutron vpn-service-create --name vpn-west --description "VPN WEST" $WEST_ROUTER_ID $WEST_SUBNET_ID
neutron ipsec-site-connection-create --name vpn_conn_west --vpnservice-id vpn-west --ikepolicy-id ikepolicy1 --ipsecpolicy-id ipsecpolicy1 --peer-address $EAST_EXT_IP --peer-id $EAST_EXT_IP --peer-cidr 10.0.2.0/24 --psk password
neutron ipsec-site-connection-list
neutron vpn-service-list
neutron vpn-service-update --admin_state_up=True
d, Check the status, ok, the status of ipsec-site-connection object should be DOWN before the data arrive in for 'auto=route' passive mode.
hua@hua-ThinkPad-T440p:/bak/openstack/devstack$ nova list
+--------------------------------------+-------------+--------+------------+-------------+---------------------------------------+
| ID | Name | Status | Task State | Power State | Networks |
+--------------------------------------+-------------+--------+------------+-------------+---------------------------------------+
| 9ce00d9d-8145-4859-aa43-539fc8efb10e | vpn-vm-east | ACTIVE | - | Running | vpn-net-east=10.0.2.2, 192.168.101.12 |
| f733855a-e768-4f59-9dad-731facdc2e4c | vpn-vm-west | ACTIVE | - | Running | vpn-net-west=10.0.3.2, 192.168.101.13 |
+--------------------------------------+-------------+--------+------------+-------------+---------------------------------------+
hua@hua-ThinkPad-T440p:/bak/openstack/devstack$ neutron vpn-service-list
+--------------------------------------+----------+--------------------------------------+--------+
| id | name | router_id | status |
+--------------------------------------+----------+--------------------------------------+--------+
| 1e9c2fa1-deb4-4d02-8907-4979ced048d0 | vpn-west | d993a249-3a4d-4c64-b33a-35363dcd83b7 | ACTIVE |
| e65a78fa-6bda-4aea-bd0f-ccb5010a4ed0 | vpn-east | efb33180-6ad2-44ce-b94a-708943d1150a | ACTIVE |
+--------------------------------------+----------+--------------------------------------+--------+
hua@hua-ThinkPad-T440p:/bak/openstack/devstack$ neutron ipsec-site-connection-list
+--------------------------------------+---------------+---------------+---------------+------------+-----------+--------+
| id | name | peer_address | peer_cidrs | route_mode | auth_mode | status |
+--------------------------------------+---------------+---------------+---------------+------------+-----------+--------+
| 81bbd115-d713-4135-8394-25aeeccc1a8c | vpn_conn_east | 192.168.101.5 | "10.0.3.0/24" | static | psk | DOWN |
| fed4a953-36d8-47a9-b439-d785144e9a11 | vpn_conn_west | 192.168.101.5 | "10.0.2.0/24" | static | psk | DOWN |
+--------------------------------------+---------------+---------------+---------------+------------+-----------+--------+
e, Check the status again after the ping data arrive in, ok, the status of ipsec-site-connection object should be ACTIVE after the data arrive in for 'auto=route' passive mode.
hua@hua-ThinkPad-T440p:~$ ssh cirros@192.168.101.13
$ ip addr show eth0 |grep eth0
2: eth0:
inet 10.0.3.2/24 brd 10.0.3.255 scope global eth0
$ ping -c 1 10.0.2.2
PING 10.0.2.2 (10.0.2.2): 56 data bytes
64 bytes from 10.0.2.2: seq=0 ttl=62 time=1.038 ms
--- 10.0.2.2 ping statistics ---
1 packets transmitted, 1 packets received, 0% packet loss
round-trip min/avg/max = 1.038/1.038/1.038 ms
hua@hua-ThinkPad-T440p:/bak/openstack/devstack$ neutron ipsec-site-connection-list
+--------------------------------------+---------------+---------------+---------------+------------+-----------+--------+
| id | name | peer_address | peer_cidrs | route_mode | auth_mode | status |
+--------------------------------------+---------------+---------------+---------------+------------+-----------+--------+
| 801509ee-8cff-4a2e-ae16-08aafb170e5b | vpn_conn_west | 192.168.101.4 | "10.0.2.0/24" | static | psk | ACTIVE |
| 81bbd115-d713-4135-8394-25aeeccc1a8c | vpn_conn_east | 192.168.101.5 | "10.0.3.0/24" | static | psk | ACTIVE |
+--------------------------------------+---------------+---------------+---------------+------------+-----------+--------+
hua@hua-ThinkPad-T440p:/bak/openstack/devstack$ neutron vpn-service-list
+--------------------------------------+----------+--------------------------------------+--------+
| id | name | router_id | status |
+--------------------------------------+----------+--------------------------------------+--------+
| 1e9c2fa1-deb4-4d02-8907-4979ced048d0 | vpn-west | d993a249-3a4d-4c64-b33a-35363dcd83b7 | ACTIVE |
| e65a78fa-6bda-4aea-bd0f-ccb5010a4ed0 | vpn-east | efb33180-6ad2-44ce-b94a-708943d1150a | ACTIVE |
+--------------------------------------+----------+--------------------------------------+--------+
3, The principle behind
a, route info added by strongswan process On East and West
hua@hua-ThinkPad-T440p:~$ sudo ip netns exec qrouter-efb33180-6ad2-44ce-b94a-708943d1150a ip route list table 220
10.0.3.0/24 via 192.168.101.5 dev qg-1dd8df9a-bf proto static src 10.0.2.1
hua@hua-ThinkPad-T440p:~$ sudo ip netns exec qrouter-d993a249-3a4d-4c64-b33a-35363dcd83b7 ip route list table 220
10.0.2.0/24 via 192.168.101.4 dev qg-12ccbd3c-6d proto static src 10.0.3.1
b, iptables rulles added by strongswan vpnaas driver On East and West
hua@hua-ThinkPad-T440p:~$ sudo ip netns exec qrouter-efb33180-6ad2-44ce-b94a-708943d1150a iptables -nL -t nat |grep ipsec
ACCEPT all -- 10.0.2.0/24 10.0.3.0/24 policy match dir out pol ipsec
hua@hua-ThinkPad-T440p:~$ sudo ip netns exec qrouter-d993a249-3a4d-4c64-b33a-35363dcd83b7 iptables -nL -t nat |grep ipsec
ACCEPT all -- 10.0.3.0/24 10.0.2.0/24 policy match dir out pol ipsec
c, debug
hua@hua-ThinkPad-T440p:~$ sudo /usr/local/bin/neutron-rootwrap /etc/neutron/rootwrap.conf ip netns exec qrouter-efb33180-6ad2-44ce-b94a-708943d1150a neutron-vpn-netns-wrapper --mount_paths=/etc:/opt/stack/data/neutron/ipsec/efb33180-6ad2-44ce-b94a-708943d1150a/etc,/var/run:/opt/stack/data/neutron/ipsec/efb33180-6ad2-44ce-b94a-708943d1150a/var/run --cmd=ipsec,status
Command: ['mount', '--bind', '/opt/stack/data/neutron/ipsec/efb33180-6ad2-44ce-b94a-708943d1150a/etc', '/etc'] Exit code: 0 Stdout: Stderr: Command: ['mount', '--bind', '/opt/stack/data/neutron/ipsec/efb33180-6ad2-44ce-b94a-708943d1150a/var/run', '/var/run'] Exit code: 0 Stdout: Stderr: Command: ['ipsec', 'status'] Exit code: 0 Stdout: Routed Connections:
81bbd115-d713-4135-8394-25aeeccc1a8c{1}: ROUTED, TUNNEL
81bbd115-d713-4135-8394-25aeeccc1a8c{1}: 10.0.2.0/24 === 10.0.3.0/24
Security Associations (1 up, 0 connecting):
81bbd115-d713-4135-8394-25aeeccc1a8c[2]: ESTABLISHED 11 minutes ago, 192.168.101.4[192.168.101.4]...192.168.101.5[192.168.101.5]
81bbd115-d713-4135-8394-25aeeccc1a8c{1}: INSTALLED, TUNNEL, ESP SPIs: cbabdd51_i cfb67c36_o
81bbd115-d713-4135-8394-25aeeccc1a8c{1}: 10.0.2.0/24 === 10.0.3.0/24
Stderr:
hua@hua-ThinkPad-T440p:~$ sudo /usr/local/bin/neutron-rootwrap /etc/neutron/rootwrap.conf ip netns exec qrouter-d993a249-3a4d-4c64-b33a-35363dcd83b7 neutron-vpn-netns-wrapper --mount_paths=/etc:/opt/stack/data/neutron/ipsec/d993a249-3a4d-4c64-b33a-35363dcd83b7/etc,/var/run:/opt/stack/data/neutron/ipsec/d993a249-3a4d-4c64-b33a-35363dcd83b7/var/run --cmd=ipsec,status
Command: ['mount', '--bind', '/opt/stack/data/neutron/ipsec/d993a249-3a4d-4c64-b33a-35363dcd83b7/etc', '/etc'] Exit code: 0 Stdout: Stderr: Command: ['mount', '--bind', '/opt/stack/data/neutron/ipsec/d993a249-3a4d-4c64-b33a-35363dcd83b7/var/run', '/var/run'] Exit code: 0 Stdout: Stderr: Command: ['ipsec', 'status'] Exit code: 0 Stdout: Routed Connections:
801509ee-8cff-4a2e-ae16-08aafb170e5b{1}: ROUTED, TUNNEL
801509ee-8cff-4a2e-ae16-08aafb170e5b{1}: 10.0.3.0/24 === 10.0.2.0/24
Security Associations (1 up, 0 connecting):
801509ee-8cff-4a2e-ae16-08aafb170e5b[1]: ESTABLISHED 11 minutes ago, 192.168.101.5[192.168.101.5]...192.168.101.4[192.168.101.4]
801509ee-8cff-4a2e-ae16-08aafb170e5b{1}: INSTALLED, TUNNEL, ESP SPIs: cfb67c36_i cbabdd51_o
801509ee-8cff-4a2e-ae16-08aafb170e5b{1}: 10.0.3.0/24 === 10.0.2.0/24
Stderr:
4, configurations generated by strongswan vpnaas driver
hua@hua-ThinkPad-T440p:~$ cat /opt/stack/data/neutron/ipsec/efb33180-6ad2-44ce-b94a-708943d1150a/etc/ipsec.conf
# Configuration for vpn-east
config setup
conn %default
ikelifetime=60m
keylife=20m
rekeymargin=3m
keyingtries=1
authby=psk
keyexchange=ikev2
mobike=no
conn 81bbd115-d713-4135-8394-25aeeccc1a8c
left=192.168.101.4
leftsubnet=10.0.2.0/24
leftid=192.168.101.4
leftfirewall=yes
right=192.168.101.5
rightsubnet=10.0.3.0/24
rightid=192.168.101.5
auto=route
hua@hua-ThinkPad-T440p:~$ cat /opt/stack/data/neutron/ipsec/d993a249-3a4d-4c64-b33a-35363dcd83b7/etc/ipsec.conf
# Configuration for vpn-west
config setup
conn %default
ikelifetime=60m
keylife=20m
rekeymargin=3m
keyingtries=1
authby=psk
keyexchange=ikev2
mobike=no
conn 801509ee-8cff-4a2e-ae16-08aafb170e5b
left=192.168.101.5
leftsubnet=10.0.3.0/24
leftid=192.168.101.5
leftfirewall=yes
right=192.168.101.4
rightsubnet=10.0.2.0/24
rightid=192.168.101.4
auto=route
hua@hua-ThinkPad-T440p:~$ cat /opt/stack/data/neutron/ipsec/efb33180-6ad2-44ce-b94a-708943d1150a/etc/ipsec.secrets
# Configuration for vpn-east
192.168.101.4 192.168.101.5 : PSK "password"
hua@hua-ThinkPad-T440p:~$ cat /opt/stack/data/neutron/ipsec/d993a249-3a4d-4c64-b33a-35363dcd83b7/etc/ipsec.secrets
# Configuration for vpn-west
192.168.101.5 192.168.101.4 : PSK "password"
hua@hua-ThinkPad-T440p:~$ cat /opt/stack/data/neutron/ipsec/d993a249-3a4d-4c64-b33a-35363dcd83b7/etc/strstrongswan.conf
charon {
load_modular = yes
plugins {
include strongswan.d/charon/*.conf
}
}
hua@hua-ThinkPad-T440p:~$ cat /opt/stack/data/neutron/ipsec/d993a249-3a4d-4c64-b33a-35363dcd83b7/etc/strongswan.conf
charon {
load_modular = yes
plugins {
include strongswan.d/charon/*.conf
}
}
include strongswan.d/*.conf