In this tutorial, We will demo how to use Fortigate and Alibaba CEN-TR to secure enterprise east-west and north-south traffic . The term east-west traffic refers to the traffic between two VPCs, VPC-A and VPC-B. The term north-south traffic refers to the traffic from VPC-A or VPC-B outbound to the internet. VPC-A and VPC-B itself does not have internet access, instead, it will use a fortigate in securehub VPC (HUBVPC) as the exit to the internet. so all traffic can be inspected by Fortigate. Fortigate is located at a VPC named HUBVPC. where it inspect all the traffic from VPC-A /VPC-B. no matter the destination is internet or other VPCs. Fortigate is a Next Generation Firewall, it will be able to inspect L3 to L7 traffic. The CEN-TR connects all VPCs together with an alicloud internal network to offer a high quality VPC peering network . As all VPCs and CEN-TR in the same region. so the traffic between All VPCs is free without charge.
For people who are not familiar with Fortinet Fortigate firewall and Alibaba CEN-TR , please refer to https://www.fortinet.com/content/dam/fortinet/assets/data-sheets/FortiGate_VM_AliCloud.pdf and https://www.alibabacloud.com/blog/build-an-enterprise-level-private-network-with-cloud-enterprise-network---transit-router-cen-tr_597397
Network Design
HUBVPC
HUBVPC has 3 vswitches in each AZ zone. one switch called landing vswitch is used for connect CEN-TR, one switch called internal for routing VPC traffic to fortigate, East-West traffic will be routed into HUBVPC and then route to fortigate via this internal routing table. The Rest routing table is bound to default vswitch where it has internet access via VPC router. Fortigate port 1 is connected to this switch.
Fortigate
Fortigate is a ECS VM in HubVPC, the port1 on fortigate is associated with VPC default switch, so it is the exit to internet, port2 is an ENI on internal vswitch. it will take the traffic from internal vswitch, so the traffic from internal vswitch can arrive fortigate.
VPC-A and VPC-B
VPC-A and VPC-B are the customer workload VPC. It has ECS in both VPCs. the ECS in both VPCs will communicate with each other also they will need to access the internet. VPC-A and VPC-B only have one default vswitch and default routing table. It routes all the traffic to CEN-TR. CEN-TR in turn routes all the traffic to HUBVPC for Fortigate to do inspection.
In this tutorial, we also put a terraform script for creating all the resources ( but you can also use an alicloud console GUI instead). after bringing up all the resources with all configuration. the ECS in VPCA, VPCB can communicate with each other and both will be able to reach the internet. We will verify this by using ping and curl on VM in VPC-A and VPC-B. Then we will check fortigate fortiview for traffic activities.
All the scripts are also tested on alibaba computenest service.
Let's start .
Create CEN
Create a CEN instance, you can create CEN instance in console or use terraform script below
resource "alicloud_cen_instance" "default" { cen_instance_name = "${var.cen_instance_name}-${random_string.random.result}"description = "terraform01"}
A CEN instance created accordingly. The default version is “Enterprise”.
Create CEN-TR
Create a TR router in region cn-hongkong associated with cen instance
cen_id = alicloud_cen_instance.default.id}
A CEN-TR router is created . by default. it has no Bandwidth Plan associated , however, 1kbps traffic is allowed for testing. which is enough for us to verify the connectivity.
Create CEN-TR routing table
CEN-TR is a virtual router, just like any router can hold a routing table, in our tutorial, CEN-TR will require to create 2 routing tables. one routing table is for VPC_A and VPC-B to associate , another routing table is for HUBVPC to associate.
The routing table “default_to_hub_for_spoke_vpc” with name “tf-default “ is for VPC_A and VPC_B to associate .
The routing table to_spoke_vpc is for hubvpc to associate.
`resource "alicloud_cen_transit_router_route_table" "default_to_hub_for_spoke_vpc" {
`
transit_router_id = alicloud_cen_transit_router.default.transit_router_id
transit_router_route_table_name="${var.cen_tr_name}-${var.cen_tr_table_name_default}"
}
resource "alicloud_cen_transit_router_route_table" "to_spoke_vpc" {
transit_router_id = alicloud_cen_transit_router.default.transit_router_id
transit_router_route_table_name="${var.cen_tr_name}-${var.cen_tr_table_name_to_spoke_vpc}"
}
you will see another default routing table with type “System”. This is the default one. in this tutorial. we are not going to use this one. so just leave it there. above you shall see two routing tables (tf-to_spoke_vpc) and (tf-default) have been created.
Attach all VPCs to CEN-TR
As all VPCs are in the same hongkong region, and we only use one CEN-TR router, we just attach all VPCs to the same CEN-TR router.
Attach hubvpc to CEN-TR
resource "alicloud_cen_transit_router_vpc_attachment" "hubvpc" {
cen_id = alicloud_cen_instance.default.id
transit_router_id = alicloud_cen_transit_router.default.transit_router_id
vpc_id = alicloud_vpc.vpc.id
zone_mappings {
data.alicloud_cen_transit_router_available_resource.default.zones.0.master_zones.0
zone_id = "cn-hongkong-b"
vswitch_id = alicloud_vswitch.landing_for_cen_a_0.id
}
zone_mappings {
data.alicloud_cen_transit_router_available_resource.default.zones.0.slave_zones.0
zone_id = "cn-hongkong-c"
vswitch_id = alicloud_vswitch.landing_for_cen_c_2.id
}
transit_router_attachment_name = "${var.transit_router_attachment_name}-${alicloud_vpc.vpc.name}"
transit_router_attachment_description = "terraform"
}
Above we just hard-coded zone b and zone C for TR. if you found your zone for TR is not C and B, please change it accordingly or use zone id data source in terraform to get zonemapping.
Attach spoke VPCs to CEN-TR
resource "alicloud_cen_transit_router_vpc_attachment" "spoke_a" {
depends_on =[alicloud_cen_transit_router_vpc_attachment.hubvpc]
cen_id = alicloud_cen_instance.default.id
transit_router_id = alicloud_cen_transit_router.default.transit_router_id
vpc_id = alicloud_vpc.vpc_a.id
zone_mappings {
data.alicloud_cen_transit_router_available_resource.default.zones.0.master_zones.0
zone_id = "cn-hongkong-b"
vswitch_id = alicloud_vswitch.vswitch_a_zone_a.id
}
zone_mappings {
data.alicloud_cen_transit_router_available_resource.default.zones.0.slave_zones.0
zone_id = "cn-hongkong-c"
vswitch_id = alicloud_vswitch.vswitch_a_zone_b.id
}
transit_router_attachment_name = "${var.transit_router_attachment_name}-${alicloud_vpc.vpc_a.name}"
transit_router_attachment_description = "terraform"
}
resource "alicloud_cen_transit_router_vpc_attachment" "spoke_b" {
depends_on = [alicloud_cen_transit_router_vpc_attachment.spoke_a]
cen_id = alicloud_cen_instance.default.id
transit_router_id = alicloud_cen_transit_router.default.transit_router_id
vpc_id = alicloud_vpc.vpc_b.id
`zone_mappings {
data.alicloud_cen_transit_router_available_resource.default.zones.0.master_zones.0
zone_id = "cn-hongkong-b"
vswitch_id = alicloud_vswitch.vswitch_b_zone_a.id
}
`
`zone_mappings {
data.alicloud_cen_transit_router_available_resource.default.zones.0.slave_zones.0
zone_id = "cn-hongkong-c"
vswitch_id = alicloud_vswitch.vswitch_b_zone_b.id
}
`
`transit_router_attachment_name = "${var.transit_router_attachment_name}-${alicloud_vpc.vpc_b.name}"
transit_router_attachment_description = "terraform"`
}
TR attaches to vswitch in both zone B and zone C, so we need to use zone-mapping to map TR zone and VPC zone. here we also use zone B and zone C.
so far we have created CEN,TR,attached (create connection) all VPCs to TR, and also created two custom routing tables on CEN-TR.
Let's take a look at the console for what we have done.
you now have one CEN instance and one CEN-TR router
Your TR is attached with 3 VPCs and has 2 custom routing tables and one system routing table.
Next we will need to associate VPCs to the TR routing table as well as to populate the routing table on CEN-TR.
In our example, traffic from HUBVPC will need to use the CEN-TR routing table “tf-to-spoke_vpc” , the routing table lookup will be done on this routing table. for traffic that from spoke vpc vpc_a and vpc-b has to use routing table “tf-default” as traffic from vpc-a and vpc-b will consult this routing table to reach fortigate.
to associate HUBVPC to routing table “default_hub_vpc”
resource "alicloud_cen_transit_router_route_table_association" "default_hub_vpc" {
transit_router_route_table_id = alicloud_cen_transit_router_route_table.to_spoke_vpc.transit_router_route_table_id
transit_router_attachment_id = alicloud_cen_transit_router_vpc_attachment.hubvpc.transit_router_attachment_id
}
to associate spoke vpc to routing table “tr-default”
resource "alicloud_cen_transit_router_route_table_association" "associate_spoke_vpc_a_to_tr_rtb_default" {
depends_on =[alicloud_cen_transit_router_route_table_association.associate_spoke_vpc_b_to_tr_rtb_default]
transit_router_route_table_id = alicloud_cen_transit_router_route_table.default_to_hub_for_spoke_vpc.transit_router_route_table_id
transit_router_attachment_id = alicloud_cen_transit_router_vpc_attachment.spoke_a.transit_router_attachment_id
}
resource "alicloud_cen_transit_router_route_table_association" "associate_spoke_vpc_b_to_tr_rtb_default" {
depends_on = [time_sleep.wait_60_seconds]
transit_router_route_table_id = alicloud_cen_transit_router_route_table.default_to_hub_for_spoke_vpc.transit_router_route_table_id
transit_router_attachment_id = alicloud_cen_transit_router_vpc_attachment.spoke_b.transit_router_attachment_id
}
Next We will need to populate the CEN-TR routing table.
for tf-to-spoke-vpc. It will require route entry from vpc-a and vpc-b and hub-vpc. so we will need to propagate all VPCs route entries into this routing table.
resource "alicloud_cen_transit_router_route_table_propagation" "default_for_hubvpc_to_spoke_vpc" {
transit_router_route_table_id = alicloud_cen_transit_router_route_table.to_spoke_vpc.transit_router_route_table_id
transit_router_attachment_id = alicloud_cen_transit_router_vpc_attachment.hubvpc.transit_router_attachment_id
}
resource "alicloud_cen_transit_router_route_table_propagation" "propagte_spoke_a_vpc_route_to_spoke_vpc_rt" {
transit_router_route_table_id = alicloud_cen_transit_router_route_table.to_spoke_vpc.transit_router_route_table_id
transit_router_attachment_id = alicloud_cen_transit_router_vpc_attachment.spoke_a.transit_router_attachment_id
}
resource "alicloud_cen_transit_router_route_table_propagation" "propagte_spoke_b_vpc_route_to_spoke_vpc_rt" {
transit_router_route_table_id = alicloud_cen_transit_router_route_table.to_spoke_vpc.transit_router_route_table_id
transit_router_attachment_id = alicloud_cen_transit_router_vpc_attachment.spoke_b.transit_router_attachment_id
}
for tf-default. it only need one default route to point to hubvpc via TR. so it does not need to propagate any route into this routing table.
Summary
Now, We have all three VPCs attached to CEN-TR, and two routing tables created on this CEN-TR and associated with VPCs , also we propagate route entries from VPCs into TR routing tables. so TR knows how to get to the destination. The TR-default routing table does not have any route entries yet as we are not propate any route into this routing table. TR tf-to_spoke_vpc have all the routes propagated from 3 VPCs.
on the TR router tf-default routing table. We will need to add a static route to tell tr-default all traffic shall be sent to hubVPC via TR attachment (to HubVPC).
`variable "rt_default" {
default = "0.0.0.0/0"
}
`
// for config a default route to tr hub-vpc on tf-default for spoke vpc to use
resource "alicloud_cen_transit_router_route_entry" "tr_cen_tf_default_router_entry" {
transit_router_route_table_id = alicloud_cen_transit_router_route_table.default_to_hub_for_spoke_vpc.transit_router_route_table_id
transit_router_route_entry_destination_cidr_block = var.rt_default
transit_router_route_entry_next_hop_type = "Attachment"
transit_router_route_entry_name = "terraform-tf-default-0.0.0.0"
transit_router_route_entry_description = "terraform-tf-default-0.0.0.0"
transit_router_route_entry_next_hop_id = alicloud_cen_transit_router_vpc_attachment.hubvpc.transit_router_attachment_id
}
When traffic from spoke vpc reaches TR, TR will use the tf-default routing table, 0.0.0.0/0 is the only entry in this routing table. so all traffic will be sent to hubvpc via Next-hop tr-atta-hubvpc-FG-AP-ljet.
Next We need to configure VPCs routing table to tell VPC to send traffic into the TR router. spoke VPC-a and VPC-b which is quite simple. We only need to tell VPC-a and VPC-b to use the default route to reach TR.
the terraform script for add route entry in VPC-a and VPC-b
`variable "default0" {
default = "0.0.0.0/0"
}
`
`resource "alicloud_route_entry" "spoke_a_default" {
route_table_id = alicloud_vpc.vpc_a.route_table_id
` alicloud_route_entry.spoke_vpc_default_route[0].route_table_id destination_cidrblock = var.default0nexthop_type = "Attachment" name = "tf-default_to_tr" nexthop_id = alicloud_cen_transit_router_vpc_attachment.spoke_a.transit_router_attachment_id}resource "alicloud_route_entry" "spoke_b_default" {alicloud_route_entry.spoke_vpc_default_route[0].route_table_id
`route_table_id = alicloud_vpc.vpc_b.route_table_id
destination_cidrblock = var.default0
nexthop_type = "Attachment"
name = "tf-default_to_tr"`nexthop_id = alicloud_cen_transit_router_vpc_attachment.spoke_b.transit_router_attachment_id}
from console, we can config VPC routing table either use VPC menu or in CEN-TR menu, use “Network instance Route Table” menu, here we show how to use “network instance Route table” under CEN-TR , first we filter spoke VPC VPC-A (192.168.10.0/20), add click Add Route Entry to add a static route. do the same for VPC-b.
Do the same for other spoke VPC.
Next we need to configure the hub vpc routing table.
In hubvpc, things are slightly complicated. As in hubvpc, we have 3 routing tables. (2 custom routing tables and 1 system default).
one routing table is for fortigate internal network. Fortigate port2 is on this network.
System default routing table is just default. It routes traffic to the internet by default. Fortigate port 1 is on this network.
One routing table is for CEN-TR to land traffic. this routing table will need to route all traffic to port2 ENI ,so all traffic from TR will go into fortigate.
The landing routing table is associated with the landing switch, if the landing switch is connecting with TR via zone B and zone C, then make sure the landing switch is in zone B and zone C.
Let's create a TR landing routing table.
`variable "tr_landing_route_entry" {
default = "0.0.0.0/0"
}
`
`resource "alicloud_route_table" "tr_landing_rt" {
depends_on = [time_sleep.wait_120_seconds]
count = 1
`
`vpc_id = alicloud_vpc.vpc.id
route_table_name = "Tr-landing-${random_string.random_name_post.result}-${count.index}"
description = "TR-landing routing table"
`}
`resource "alicloud_route_entry" "spoke_vpc_default_route" {
count = 1
route_table_id = alicloud_route_table.tr_landing_rt[count.index].id
destination_cidrblock = var.tr_landing_route_entry //Default is 0.0.0.0/0
nexthop_type = "NetworkInterface"
`
`name = alicloud_network_interface.PrimaryFortiGateInterface1.id
nexthop_id = alicloud_network_interface.PrimaryFortiGateInterface1.id
`}
`resource "alicloud_route_table_attachment" "tr_landing_attachment" {
depends_on = [alicloud_route_table.tr_landing_rt]
count = 1
vswitch_id = alicloud_vswitch.landing_for_cen_a_0.id
`route_table_id = alicloud_route_table.tr_landing_rt[count.index].id}
`resource "alicloud_route_table_attachment" "tr_landing_attachment_1" {
depends_on = [alicloud_route_table.tr_landing_rt]
count = 1
vswitch_id = alicloud_vswitch.landing_for_cen_b_1.id
route_table_id = alicloud_route_table.tr_landing_rt[count.index].id
`}
`resource "alicloud_route_table_attachment" "tr_landing_attachment_2" {
depends_on = [alicloud_route_table.tr_landing_rt]
count = 1
vswitch_id = alicloud_vswitch.landing_for_cen_c_2.id
route_table_id = alicloud_route_table.tr_landing_rt[count.index].id
`}
On this hubvpc landing routing table. we will need to add a default route (0.0.0.0) to let VPC any traffic coming into this vswitch, lookup TR landing routing table , use default to reach next hop eni port2. so all traffic that from TR will be sent to VPC vswitch “internal”.
`variable "tr_landing_route_entry" {
default = "0.0.0.0/0"
`}
`resource "alicloud_route_table" "tr_landing_rt" {
depends_on = [time_sleep.wait_120_seconds]
count = 1
vpc_id = alicloud_vpc.vpc.id
route_table_name = "Tr-landing-${random_string.random_name_post.result}-${count.index}"
description = "TR-landing routing table"
`}
`resource "alicloud_route_entry" "spoke_vpc_default_route" {
count = 1
route_table_id = alicloud_route_table.tr_landing_rt[count.index].id
destination_cidrblock = var.tr_landing_route_entry //Default is 0.0.0.0/0
nexthop_type = "NetworkInterface"
name = alicloud_network_interface.PrimaryFortiGateInterface1.id
nexthop_id = alicloud_network_interface.PrimaryFortiGateInterface1.id
}
`
on hubvpc VPC custom internal routing table. we need to add a static route 192.168.0.0/19 with nexthop to TR. this is for when traffic exit from fortigate to VPC. VPC has to know for traffic that with destination 192.168.0.0/19 , send to TR-attachment towards TR. the next hop type is forwarding router.
So far, we configured the hub VPC route to send traffic into and handle traffic that leaves the fortigate . but we have not configured the fortigate yet. fortigate take the traffic from tr-landing vswitch and apply security policy and then send it back to VPC vswitch internal.
on fortigate. we have to configure a static route to let fortigate route traffic that destined to 192.168.0.0/19 to port2 ,so the traffic can enter TR as port2 is on subnet where it has nexthop TR to reach 192.168.0.0/19.
so let’s configure fortigate. On Fortigate we need to configure both the routing table and firewall policy.
above are the all configuration.
Let’s verify the result.
ssh into fortigate, then use “execute ssh 192.168.10.100” to login into VM in spoke VPC-A. the VM shall able to reach it’s VPC-B router (192.168.20.253) and VM in VPC-B (192.168.20.100).
Because traffic from VPC-A VM to VPC-B VM is via Fortigate via Port2, so
From Fortigate Fortiview, you will be able to see the traffic between two VMs in different VPCs.
Config Firewall policy on fortigate for North-South traffic.
For VPC-A and VPC-B internet bound traffic. VM on VPC-A and VPC-B does not have local internet access. it only has default route to CEN-TR, CEN-TR will lookup TF-default routing table and send traffic to hubvpc. Hubvpc will lookup TR-landing routing table and send traffic to fortigate port2 eni . so traffic arrive fortigate. once traffic arrive fortigate. fortigate will lookup the routing table , it will find that to internet the nexthop is port1 on fortigate. so next-hop , the firewall policy will be checked to see whether this is allowed. obviously, we have not configured firewall policy yet. so let’s config it.
`config firewall policy
edit 2
set name "port2_to_port1"
set srcintf "port2"
set dstintf "port1"
set srcaddr "all"
set dstaddr "all"set action accept
set schedule "always"
set service "ALL"
set utm-status enable
set ssl-ssh-profile "certificate-inspection"
set av-profile "default"
set webfilter-profile "default"
set ips-sensor "default"
set application-list "default"
set logtraffic all
set nat enable`
We have to enable NAT, and we enable UTM features as well. so the traffic from VM in VPC-A and VPC-B will be inspected by fortigate.
for example, when I issue curl -I www.google.com/ls.exe on VM in VPA-A
The fortigate will block this as this is unrated traffic according fortigate UTM policy.
Summary
Above we showed how we use Fortigate to inspect traffic between VPCs and VPC to the internet, Alibaba CEN-TR function as a virtual router and connect all VPCs together and routing traffic between VPCs.
This demo case is also available on alibaba computenest platform. you can go and try them and find all the terraform scripts as well.
21 posts | 12 followers
FollowJawnLim - April 24, 2023
Marketplace - August 17, 2020
Marketplace - August 18, 2020
vincentsiu - September 5, 2023
Alibaba Clouder - October 12, 2020
Alibaba Clouder - April 19, 2021
21 posts | 12 followers
Follow
PrivateLink
Connect your VPCs to services in other VPCs through secure, reliable, and private connections.
Learn More
VPN Gateway
VPN Gateway is an Internet-based service that establishes a connection between a VPC and your on-premise data center.
Learn More
VPC
A virtual private cloud service that provides an isolated cloud network to operate resources in a secure environment.
Learn More
Alibaba Cloud PrivateZone
Alibaba Cloud DNS PrivateZone is a Virtual Private Cloud-based (VPC) domain name system (DNS) service for Alibaba Cloud users.
Learn MoreMore Posts by Marketplace