Cloud Notes: AWS Direct Connect

Intro

AWS Direct Connect (DX) is a service that allows for a dedicated connection between your network and AWS.

Features

AWS Direct Connect has the following key features:

• Delivers consistent, low-latency performance.
• A Dedicated connection, connects you directly to AWS via a 1, 10 or 100 Gbps link.
• Hosted AWS partner connections provide speeds of 50 Mbps up to 10 Gbps and connect to AWS via the partners port. Example Providers: Megaport and Equinix.
• MACsec is available for physical link level encryption between devices.
• IPSec encryption is used for end-to-end encryption.

Direct Connect Architecture

The following diagram shows a typical DX architecture.

The following points describe the above diagram.

• The customer on-premises environment is connected to a customer router at the DX Location via a WAN connection.
• The customer router is connected to the DX router via a cross-connect.
• 3x VLANs are configured on the DX router and each VLAN is mapped to a VIF.
• The Green VLAN (101) is mapped to a Transit VIF.
• The Blue VLAN (102) is mapped to a Private VIF.
• The Pink VLAN (103) is mapped to a Public VIF.
• A BGP session is established for each VIF to the Customer router.
• The Transit VIF is connected to a Transit Gateway via a DX Gateway.
• The Private VIF is connected to a VPC via a VGW.
• The Public VIF is connected to an AWS Public Service via a Public IP.

Virtual Interfaces

Virtual Interfaces (VIFs) are used to connect your network to AWS via a DX Location. A VIF is a combination of a BGP peering session and a VLAN.

There are 3 types of VIFs: Private, Public and Transit.

Private VIF

Private VIFs are used to connect to AWS services using Private IPs. IE: VPC based services that do not have Elastic IPs.

The following are some important points to note about Private VIFs:

• Private VIFs can only connect to a VGW in the same region as the DX Location.
• By default Private VIFs are NOT encrypted.
• MTU sizes from 1500 to 9001 bytes are supported (Jumbo Frames).
• Route propogation is enabled by default when terminated to a VGW.
• BGP ASN can be from the public or private ASN range (64512 - 65535).
• AWS advertises the VPC CIDR(s) and the BGP Peer IPs.
• The Customer can advertise a maximum of 100 prefixes.
• Both IPv4 and IPv6 are supported via a seperate BGP peering.

Public VIF

Public VIFs are used to connect to AWS services using Public IPs. IE: AWS public services such as S3 and SQS or VPC based services that have Elastic IPs.

The following are some important points to note about Public VIFs:

• There is no direct access to Private VPC services.
• Can access all Regions public zone services (Excluding China).
• AWS advertises ALL AWS Public IP ranges.
• The Customer can ONLY advertise Public IP Prefixes.
• BGP communities can be used to TAG Prefixes from BOTH AWS and the Customer.
• Customer advertised prefixes are non-transitive, they will NOT be re-advertised by AWS to external peers.
• BGP ASN can be from the public or private ASN range (64512 - 65535).

Transit VIF

Transit VIFs allow integration between AWS Direct Connect and Transit Gateway.

The following are some important points to note about Transit VIFs:

• Transit VIFs can only connect to a Transit Gateway in the same region as the DX Location.
• A TGW peering is required to connect to TGWs in other regions.

Direct Connect Gateway

A Direct Connect (DX) Gateway is a virtual router that allows Private VIFs to connect to multiple VPCs. in ANY region via either a Virtual Private Gateway (VGW) or a Transit Gateway (TGW).

The following are some important points to note about DX Gateways:

• The DX Gateway is GLOBAL resource. You can connect from any Region, to any Region globally using a DX gateway.
• The DX Gateway ONLY routes between AWS resources and the Customer Gateway (CGW).
• A single Private VIF can be associated with a DX Gateway.
• A single DX Gateway can be associated with up to 10 VGWs.
• A single Transit VIF can be associated with a DX Gateway.
• A single DX Gateway can be associated with up to 3 TGWs.
• A DX Gateway can be associated with either VGWs (Private VIFs) or a TGWs (Transit VIFs), but NOT both.
• A TGW can be associated with a maximum of 20 DX Gateways.

SiteLink

SiteLink is a is DX feature that allows you to send data between DX locations via a DX Gateway without having to route though an AWS region.

The following are some important points to note about SiteLink:

• SiteLink requires a DX Gateway.
• Direct Connects MUST be connected to the same DX Gateway.
• SiteLink is available for Dedicated and Hosted connections.
• SiteLink can be enabled on existing connections.
• SiteLink is ONLY available on Private and Transit VIFs.

The following diagram shows a typical SiteLink architecture.

The following points describe the above diagram.

• The DX Gateway is deployed in the AWS Global Zone (No Region).
• The Customer routers are connected to DX Routers at a DX location via a cross-connect.
• 2x VLANs are configured on the DX router and each VLAN is mapped to a VIF, 1x Tranist and 1x Private.
• The VIF are configured with the SiteLink option Enabled.

BGP Communities

BGP communities can be used to tag prefixes and allow administrators to signal routing policy on upstream and downstream routers.

The following are some important points to note about BGP communities:

• the AWS ASN is 7224.
• AWS DX tags incoming customer prefies with the NO_EXPORT community.
• BGP communities can be represented as 2x 16bit values in the format <BGP_ASN>:<ADMIN_ASSIGNED>
• Communites can be used to define how far a prefix is advertised within the AWS network.

Community Tag Reach

The following table shows how BGP community tags influence the routing path and how far a prefix is advertised within the AWS network from a DX location.

Bi-Directional Forwarding Detection (BFD)

Bi-Directional Forwarding Detection (BFD) is a protocol that is used to detect pysical link failures between two peers. BFD then signals the failure to the routing protocol to allow for faster convergence times.

BFD improves the convergence time of routing protocols to milliseconds rather than seconds.

BFD is automatically enabled on the AWS end, it is optionally configurable on the Customer Gateway (CGW).

Default BFD timers are:

• Interval - 300ms
• Multiplier - 3

Link Aggregation Group (LAG)

A Link Aggregation Group (LAG) is a method of bundling together multiple interfaces to form a single logical connection. In the case of DX this relates to multiple DX ports.

The following are some important points to note about LAGs:

• LAGs are used to increase the bandwidth between your network and AWS at a DX location. They DO NOT provide much benefit for redundancy.
• LAGs can be configured with a maximum of 4 ports.
• Traffic is load balanced across all ports in the LAG.
• All ports in the LAG must be the same speed.
• All ports in the LAG must be in the same DX Location.
• The minimumLinks attribute is used to define the minimum number of ports that must be active in the LAG for it to be considered healthy.

Letter of Authorization

A Letter of Authorization and Connecting Facility Assignment (LOA-CFA) is used to authorize the cross-connect between your network and AWS at a DX Location.

The LOA-CFA is generated by AWS and is available to download from the AWS Console.

DX Considerations

The following are some important considerations when using AWS Direct Connect:

• AWS Dedicated connections are ONLY available in 1, 10 or 100 Gbps speeds.
• The speed of a Dedicated connection cannot be changed once provisioned. To change speed, a new port MUST be provisioned.
• Connections are via Single-Mode Fibre ONLY. The following Transiceiver types are required:
  • 1Gbps - 1000BASE-LX (1310 nm)
  • 10 Gbps - 10GBASE-LR (1310 nm)
  • 100 Gbps - 100GBASE-LR4
• Auto-Negotiation MUST be disabled. Also, port speed and duplex MUST be set manually.
• The Customer Gateway (CGW) MUST support BGP with BGP MD5 authentication.
• MACsec can be optionally enabled, however, it is only available on 10 and 100 Gbps connections.
• Bidirectional Forwarding Detection (BFD) is automatically enabled on the AWS end, it is optionally configurable on the CGW.
• There is an hourly cost for each DX port.
• There is a outbound data transfer cost for each GigaByte (GB) of data transferred out of AWS.
• Inboud data transfer has no cost.
• Dedicated connections can have up to 50 Public/Private VIFs. and 1 Transit VIF.
• Hosted Connections can only have 1 VIF.