SD-WAN & AWS Cloud WAN: Segmentation & Cost Guide

This article provides an in-depth analysis of the technical challenges and cost implications faced by enterprises when extending their on-premises SD-WAN…

云雀SD-WAN

SD-WAN and AWS Cloud WAN Network Segmentation Deployment Guide: Challenges Analysis and Cost Optimization Strategies

Executive Summary: For enterprises operating multi-tenant environments, regulated industries, or multiple business units, maintaining strict network segmentation between SD-WAN and AWS Cloud is fundamental to meeting security, compliance, and operational requirements. Extending the on-premises SD-WAN's Virtual Routing and Forwarding (VRF) segmentation capabilities seamlessly to AWS Cloud WAN enables these segmented environments to be unified under a scalable global network. This article elaborates in detail on the implementation challenges, configuration steps, cost structure, and best practices for two core deployment models (GRE tunnel model and tunnel-less model), aiming to provide enterprises with a practical and optimizable integrated solution.

Prerequisites: Essential Pre-Deployment Assessment and Preparation

Before initiating any deployment, it is imperative to confirm that the following environment, tools, and knowledge are ready, as this is the premise for ensuring project success.

1. AWS Environment Requirements:

  • An AWS account with appropriate IAM permissions (including permissions for services like VPC, Transit Gateway, Cloud WAN).
  • Planned and pre-created Transit VPC for connectivity. This VPC needs to host SD-WAN virtual appliances (e.g., Cisco CSR 1000v, Fortinet FortiGate-VM, Palo Alto VM-Series, etc.).
  • Confirmation that the AWS region supports the AWS Cloud WAN service, and clear CIDR address planning for the Global Network and Core Network.

2. On-Premises SD-WAN Environment Requirements:

  • A fully functional SD-WAN controller (e.g., Cisco vManage, FortiManager, Palo Alto Panorama).
  • A planned VRF segmentation strategy. For example, VRF-PROD for production traffic, VRF-DEV for development traffic.
  • At least one edge device (physical or virtual) capable of establishing an IPsec tunnel with AWS.

3. Network and Routing Knowledge:

  • Proficiency in routing protocols like BGP (Border Gateway Protocol) and OSPF (Open Shortest Path First).
  • Understanding of network virtualization technologies like VRF, GRE (Generic Routing Encapsulation) tunnels, VXLAN.
  • Familiarity with the basic concepts and operations of AWS Transit Gateway (TGW) and Cloud WAN.

4. Tools and Permissions:

  • AWS CLI or Console access permissions.
  • Administrator privileges for the SD-WAN controller management interface.
  • Network diagnostic tools for verifying connectivity (e.g., ping, traceroute, BGP view commands).

Environment Preparation: Initializing AWS Cloud WAN and SD-WAN Controller

This section describes the basic configuration performed on the AWS and SD-WAN controller sides, laying the foundation for establishing segmented connectivity subsequently.

Step 1: Create AWS Cloud WAN Global Network and Core Network Policy

  1. Log in to the AWS Management Console, navigate to VPC > Cloud WAN.
  2. Click “Create global network”, enter a name (e.g., Global-Corp-Network), and record its Global Network ID upon completion.
  3. Enter the created global network, click “Create core network” in the “Core network” tab.
  4. Configure the Core Network policy document. The policy defines the logic for network segmentation, routing, etc. An example policy JSON snippet is as follows: 
    { "version": "2021.12", "core-network-configuration": { "asn-ranges": ["65500-65550"], "edge-locations": ["us-east-1", "eu-west-1"], "vpn-ecmp-support": true }, "segments": [ { "name": "Production", "require-attachment-acceptance": false }, { "name": "Development", "require-attachment-acceptance": false } ], "segment-actions": [], "attachment-policies": [] }
  5. Confirm and create the core network. Record the Core Network ID and Core Network Policy Version.

Expected Result: A core network containing two segments, Production and Development, is successfully created in Cloud WAN, with status Available.

Step 2: Configure WAN Interface and Zone on the SD-WAN Controller

Illustrated with generic configuration command examples (specific commands vary by vendor):

  1. Log in to the SD-WAN controller management interface.
  2. Navigate to the interface configuration page, identify the physical or logical interface used for connecting to AWS (e.g., ge0/0).
  3. Create or confirm the WAN Zone for AWS connectivity. Example configuration thought: 
    config system zone edit "AWS-Zone" set interface "ge0/0" next end
  4. Ensure the interface has a reachable IP address and default route configured, enabling internet access to AWS public endpoints.

Expected Result: The SD-WAN controller has defined the WAN Zone and interface pointing to AWS, and basic network connectivity is ready.

Core Operations: Deployment Steps for Two Segmentation Extension Models

Enterprises can choose one of the two main models based on requirements for network complexity, performance overhead, and operational capabilities. The following details the steps for each.

Model One: GRE Tunnel-based Connect Attachment Model (One-to-One Mapping)

This model creates an independent GRE tunnel and Connect attachment on the AWS TGW for each SD-WAN VRF, mapping to an independent segment in Cloud WAN. It provides the strictest segmentation isolation but is more complex to configure.

Step 3.1: Create Multiple Connect Attachments on AWS TGW

  1. Assuming a Transit Gateway (TGW) exists and the Transit VPC is attached. In the TGW console, create a Connect attachment for each VRF that needs to be extended. 
    # Example AWS CLI command to create a Connect attachment for the Production VRF aws ec2 create-transit-gateway-connect \ --transit-gateway-id tgw-0123456789abcdef0 \ --transport-transit-gateway-attachment-id tgw-attach-0123456789abcdef0 \ # VPC Attachment ID --options '{"Protocol": "gre"}' --tag-specifications 'ResourceType=transit-gateway-connect,Tags=[{Key=Name,Value=TGW-Connect-Prod-VRF}]'
  2. Repeat this operation for the Development VRF to create a second Connect attachment.

Expected Result: Under the Connect tab in the TGW console, two connections with status Available appear, one for production and one for development traffic.

Step 3.2: Configure Cloud WAN Attachments to Associate Connect Attachments with Segments

  1. Navigate to Cloud WAN > Core network > Attachments, click “Create attachment”.
  2. Select attachment type “Connect”.
  3. Choose the corresponding Core Network ID, segment (e.g., Production), and the previously created TGW Connect attachment.
  4. Configure tags for the attachment (e.g., Segment=Prod), which can be used for automation via attachment policies.
  5. Repeat the above steps to associate the Development Connect attachment with the Development segment.

Expected Result: The Cloud WAN attachments list shows two Connect attachments, belonging to the Production and Development segments respectively, with status Available.

Step 3.3: Configure GRE Tunnels and BGP Peering on the SD-WAN Device

This step is completed on the SD-WAN edge device (virtual or physical) CLI or via templates pushed by the controller. Below is a generic configuration idea for BGP over GRE: 

! Create a GRE tunnel interface pointing to the AWS TGW Connect endpoint interface Tunnel100 description GRE-Tunnel-to-AWS-Prod-VRF ip address 169.254.10.1 255.255.255.252 tunnel source <Local_Egress_Interface_IP>tunnel destination tunnel vrf <Local_Prod_VRF_Name>! Configure BGP, establish peering with the AWS-side endpoint of Cloud WAN router bgp <Local_ASN>! address-family ipv4 vrf neighbor 169.254.10.2 remote-as 64512 # ASN used by AWS Cloud WAN neighbor 169.254.10.2 activate neighbor 169.254.10.2 soft-reconfiguration inbound exit-address-family

Expected Result: Using the show bgp vpnv4 unicast vrf <VRF_Name> summary command, the BGP neighbor state is Established, and route prefixes are exchanged.

Model Two: Tunnel-less Multi-VPC ENI Model (One-to-Many Mapping)

This model avoids creating GRE tunnels on the TGW. Instead, it directly utilizes the VPC attachment type of AWS Cloud WAN and the multi-ENI feature to map traffic from different VRFs to different ENIs, which are then associated with Cloud WAN segments. The architecture is simpler, reducing GRE encapsulation overhead.

Step 4.1: Create Multiple ENIs in the Transit VPC, Associated with Different Security Groups/Subnets

  1. In the Transit VPC, create independent subnets and security groups for each VRF (e.g., Prod, Dev) to achieve traffic isolation.
  2. Create independent ENIs for the logical network functions (e.g., firewall, router) of each VRF. 
    # Example AWS CLI command to create an ENI for Prod VRF function aws ec2 create-network-interface \ --subnet-id subnet-0123456789abcdef0 \ # Prod dedicated subnet --groups sg-0123456789abcdef0 \ # Prod dedicated security group --description "ENI for Prod VRF routing function" --tag-specifications 'ResourceType=network-interface,Tags=[{Key=Segment,Value=Production}]'

Expected Result: Two ENIs with different security attributes and subnet associations exist in the Transit VPC, serving Prod and Dev logical functions respectively.

Step 4.2: Create a Cloud WAN VPC Attachment and Enable Multi-ENI Mapping

  1. Create a VPC type attachment in the Cloud WAN console.
  2. Select the Transit VPC.
  3. In “Advanced settings”, find Appliance Mode and enable it. This allows traffic from the same source IP to always be directed to the same ENI.
  4. Key step: In the Subnet mapping, explicitly specify the Cloud WAN segment each ENI should be associated with. This is often automated by tagging the ENIs and matching them via tags in the Cloud WAN attachment policy. For example, a policy could set that when an ENI has the tag Segment=Production, traffic from that ENI is mapped to the Production segment.

Expected Result: Cloud WAN creates a VPC attachment, and the different ENIs under it are automatically assigned to the corresponding segments based on the policy.

Step 4.3: Configure Policy-Based Routing or Route Leaking on the SD-WAN Device

Since Model Two does not use GRE tunnels, the SD-WAN device needs to route traffic destined for different cloud VRF network segments to the corresponding ENI IP addresses in the Transit VPC. This is typically done via static routes or routing protocols. 

! Configure static routes, directing Prod traffic to its dedicated ENI IP ip route vrf <Prod_VRF_Name> <AWS_Prod_Subnet_CIDR> <Prod_ENI_Private_IP>! Or via BGP, receive routes advertised by Cloud WAN via the VPC attachment, and apply route filtering policies route-map AWS-PROD-ROUTES permit 10 match community set ip next-hop <Prod_ENI_Private_IP>

Expected Result: Traffic originating from the on-premises Prod-VRF can be correctly routed to the ENI in AWS serving the Production segment, and vice versa. The path can be verified using traceroute vrf <VRF_Name> <Destination_IP>.

Common Issues and Solutions

Q1: BGP neighbors cannot establish (stuck in Active or Idle state).

A: 1. Check Network Connectivity: Ensure network reachability between the local SD-WAN device and the AWS TGW Connect endpoint or ENI IP (allow ICMP and BGP port 179). 2. Verify ASN and IP Configuration: Confirm that the remote-as in the local BGP configuration matches the ASN range configured in the Cloud WAN Core Network policy. 3. Check Security Groups and ACLs: Ensure the AWS-side security groups and network ACLs allow BGP traffic from the SD-WAN device's IP.

Q2: Routes are not exchanged from SD-WAN to Cloud WAN, or vice versa.

A: 1. Check BGP Address Family: Confirm that the neighbor is activated for the correct VRF in the BGP configuration. 2. Verify Route Filters: Check if overly strict route-map or distribute-list filters are applied. 3. Confirm Cloud WAN Policy: Check if the segment-actions and attachment-policies in the Core Network policy correctly allow route propagation.

Q3: GRE tunnel performance is below expectations, high latency or low throughput.

A: 1. Adjust MTU/MSS: GRE encapsulation increases the packet header size. Adjust MTU (typically to 1400 bytes) and MSS on the tunnel interface and path devices to avoid fragmentation. 2. Enable Hardware Encryption: If using SD-WAN devices supporting hardware acceleration, ensure features like AES-NI encryption acceleration are enabled. 3. Assess Link Quality: Use the SD-WAN controller's link quality monitoring function to check packet loss, jitter, and latency metrics of the underlying internet link.

Q4: Traffic accidentally leaks to other segments in the multi-VRF model.

A: 1. Review Segment Policy: Carefully check the Cloud WAN Core Network policy JSON to ensure no share or route operations are defined between segments via segment-actions. 2. Check Attachment Mapping: Confirm that each attachment (Connect or VPC) is explicitly and correctly associated with the intended segment, with no erroneous mapping. 3. Verify Device VRF Configuration: Ensure each interface or tunnel on the SD-WAN device is strictly bound to the correct VRF.

Q5: Total cost is higher than expected, how to optimize?

A: Cost Structure Analysis: The total cost of implementing segmentation mainly includes: 1) Initial Deployment Cost: AWS data processing fees (Cloud WAN, TGW), data transfer fees (GB), SD-WAN virtual appliance licensing fees. 2) Ongoing Operational Costs: AWS service monthly fees, network traffic costs, operational personnel costs. Optimization Strategies: For scenarios with less stringent segmentation requirements, prioritize Model Two (tunnel-less) to reduce the performance overhead and potential extra computational resource costs of GRE encapsulation. Utilize AWS Reserved Instances or Savings Plans for long-term compute and network resource purchases. Conduct detailed planning for route aggregation</