B站视频链接：https://www.bilibili.com/video/BV1Rd4y1F7Fc/
微信公众号：自刘地

一、架构图

上一篇介绍了GWLB结合Network Load Balancer的场景，这里介绍一下GWLB结合Application Load Balancer的场景，主要区别还是在于路由表的设计。另外因为ALB可以关联AWS WAF，所以在代码里面顺带为ALB关联了一个WAF策略，策略只允许来自中国地区的IP访问，来自其他国家地区的请求会被拒绝。

这个架构图高清图片我放到百度网盘了，链接：https://pan.baidu.com/s/1r5uPBSmr9e0qaliz7fdTyQ 提取码：uuqd

实验环境一共有两个VPC，左边的是业务VPC，里面有两个APP模拟HTTP的业务，EC2放在私有子网里面，在不同的AZ。

有一个面向互联网的Application Load Balancer，向公网发布了这个HTTP的服务，这样互联网上的用户，就可以通过ALB访问后面的业务。我们要做的是把这个流量，引导到防火墙上去，做安全检测。这里使用Linux的iptables来模拟防火墙。

另外，APP可以通过NAT GW上网，这个APP主动访问互联网的流量，也需要送到防火墙上去，做安全检测。所以，APP有两种流量都需要送到防火墙上去做安全检测，下面看一下APP的两种流量路径。

一、来自于互联网对ALB访问的流量路径。

1. 首先，互联网上的用户，对ALB的公有DNS发起请求，这个DNS请求，会解析到ALB的两个公网IP地址，然后流量通过ISP路由，到达AWS的IGW，然后抵达ALB。
2. 流量到达ALB之后，需要转发给APP，但是去往本VPC CIDR段的流量指向了Endpoint，所以ALB会依据路由表将流量发送到Endpoint上，这里假设解析ALB的主IP地址是Public Subnet1，所以流量会给到Endpoint1。
3. Endpoint1收到流量之后，会通过Private Link，把流量发送到GWLB上。
4. GWLB会通过GENEVE封装报文，把流量发送到防火墙。
5. 防火墙做完安全检测之后，又会把流量送回给GWLB，然后通过Private Link，送到GWLB Endpoint1。
6. 到达GWLB Endpoint1之后，匹配到local路由，流量最终发送到APP1上。

继续看一下回包流程。

1. APP1收到了来自于ALB的流量，APP1看到的源IP地址是ALB弹性接口的私有IP地址，APP1关联的路由表匹配到10.120.0.0/16，将流量发送的Endpoint1。
2. 接下来又是一样的流程，流量会经过Private Link到防火墙绕一圈再回来。
3. 流量回到GWLB Endpoint1上之后，会匹配到默认路由，将流量发送到NAT GW1。
4. 流量到NAT GW1之后，匹配到默认路由，流量通过IGW发送回客户端。

以上就是完整的从互联网对ALB发起请求和回包的流程。

二、APP1主动访问互联网的流量路径

1. APP1对互联网地址发起请求，APP1查询子网关联的路由表，匹配到默认路由，流量送到Endpoint1。
2. 流量到GWLB Endpoint1之后，通过Private Link把流量送到防火墙检测，然后再发回来。
3. Endpoint1收到流量之后，会匹配到默认路由，将流量发送到NAT GW1。
4. 流量到NAT GW1之后，匹配到默认路由，流量通过IGW发送到目的地。

继续看回包流程。

1. 互联网上的主机收到报文以后，源地址是NAT GW1的公网IP地址，报文通过ISP路由，流量到达NAT GW1。
2. 到达NAT GW1之后查询转换表项，准备将流量发送到APP1的私有IP地址，NAT GW1去往本VPC CIDR段的流量指向了Endpoint1，所以流量发送到Endpoint1。
3. 流量到GWLB Endpoint1之后，通过Private Link把流量送到防火墙检测，然后再发回来。
4. Endpoint1收到流量之后，匹配到local路由，将流量发送到APP1。

二、创建实验环境

堆栈大概需要8分钟左右的时间创建完成。

上传堆栈文件。

编辑堆栈名称，修改实例密钥。

允许创建IAM资源。

Parameters:
  EC2InstanceAmiId:
    Type: AWS::SSM::Parameter::Value<AWS::EC2::Image::Id>
    Default: '/aws/service/ami-amazon-linux-latest/amzn2-ami-hvm-x86_64-gp2'
  Environment:
    Type: String
    AllowedValues:
      - dev
      - prod
    Default: dev
  MyKeyPair:
    Description: Amazon EC2 Key Pair
    Type: AWS::EC2::KeyPair::KeyName
    Default: Global_Tokyo_KeyPair
  WebServerPort:
    Description: Apache Http Server Port
    Type: String
    Default: 8443
    AllowedValues:
      - 8443
      - 8888
      - 8088

Resources:

  BastionSsmRole:
    Type: AWS::IAM::Role
    Properties:
      AssumeRolePolicyDocument:
        Statement:
          - Effect: Allow
            Principal:
              Service:
                - ec2.amazonaws.com
            Action:
              - 'sts:AssumeRole'
      Path: /

  BastionSsmPolicy:
    Type: AWS::IAM::Policy
    Properties:
      PolicyName: PrivatelianceInstanceAccess
      PolicyDocument:
        Statement:
          - Effect: Allow
            Action:
              - ssm:DescribeAssociation
              - ssm:GetDeployablePatchSnapshotForInstance
              - ssm:GetDocument
              - ssm:DescribeDocument
              - ssm:GetManifest
              - ssm:GetParameter
              - ssm:GetParameters
              - ssm:ListAssociations
              - ssm:ListInstanceAssociations
              - ssm:PutInventory
              - ssm:PutComplianceItems
              - ssm:PutConfigurePackageResult
              - ssm:UpdateAssociationStatus
              - ssm:UpdateInstanceAssociationStatus
              - ssm:UpdateInstanceInformation
            Resource: "*"
          - Effect: Allow
            Action:
              - ssmmessages:CreateControlChannel
              - ssmmessages:CreateDataChannel
              - ssmmessages:OpenControlChannel
              - ssmmessages:OpenDataChannel
            Resource: "*"
          - Effect: Allow
            Action:
              - ec2messages:AcknowledgeMessage
              - ec2messages:DeleteMessage
              - ec2messages:FailMessage
              - ec2messages:GetEndpoint
              - ec2messages:GetMessages
              - ec2messages:SendReply
            Resource: "*"
      Roles:
        - !Ref BastionSsmRole

  BastionSsmProfile:
    Type: AWS::IAM::InstanceProfile
    Properties:
      Path: /
      Roles:
        - !Ref BastionSsmRole

#=========================================SecVpc========================================#
# 创建SecVpc
  SecVpc:
    Type: AWS::EC2::VPC
    Properties:
      CidrBlock: 10.100.10.0/16
      EnableDnsSupport: 'true'
      EnableDnsHostnames: 'true'
      Tags:
       - Key: Name
         Value: !Sub ${AWS::StackName}-SecVpc

# 创建IGW并且关联到VPC
  SecVpcIGW:
    Type: "AWS::EC2::InternetGateway"
    Properties:
      Tags:
        - Key: Name
          Value: !Sub ${AWS::StackName}-SecVpcIGW

  SecVpcAttachIgw:
    Type: "AWS::EC2::VPCGatewayAttachment"
    Properties:
      VpcId: !Ref SecVpc
      InternetGatewayId: !Ref SecVpcIGW

#---------------------------SecVpc创建6个子网-------------------------------------#

# SecVpc AZ1内创建公有子网
  SecVpcAz1PublicSubnet:
    Type: AWS::EC2::Subnet
    Properties:
      VpcId: !Ref SecVpc
      CidrBlock: 10.100.10.0/24
      AvailabilityZone:
        Fn::Select:
          - 0
          - Fn::GetAZs: ""
      Tags:
      - Key: Name
        Value: !Sub ${AWS::StackName}-SecVpc-AZ1-Public-Subnet

# SecVpc AZ2内创建公有子网
  SecVpcAz2PublicSubnet:
    Type: AWS::EC2::Subnet
    Properties:
      VpcId: !Ref SecVpc
      CidrBlock: 10.100.20.0/24
      AvailabilityZone:
        Fn::Select:
          - 1
          - Fn::GetAZs: ""
      Tags:
      - Key: Name
        Value: !Sub ${AWS::StackName}-SecVpc-AZ2-Public-Subnet

# SecVpc AZ1内创建私有子网
  SecVpcAz1PrivateSubnet:
    Type: AWS::EC2::Subnet
    Properties:
      VpcId: !Ref SecVpc
      CidrBlock: 10.100.30.0/24
      AvailabilityZone:
        Fn::Select:
          - 0
          - Fn::GetAZs: ""
      Tags:
      - Key: Name
        Value: !Sub ${AWS::StackName}-SecVpc-AZ1-Private-Subnet

# SecVpc AZ2内创建私有子网
  SecVpcAz2PrivateSubnet:
    Type: AWS::EC2::Subnet
    Properties:
      VpcId: !Ref SecVpc
      CidrBlock: 10.100.40.0/24
      AvailabilityZone:
        Fn::Select:
          - 1
          - Fn::GetAZs: ""
      Tags:
      - Key: Name
        Value: !Sub ${AWS::StackName}-SecVpc-AZ2-Private-Subnet


# SecVpc AZ1内创建TGW子网
  SecVpcAz1TgwSubnet:
    Type: AWS::EC2::Subnet
    Properties:
      VpcId: !Ref SecVpc
      CidrBlock: 10.100.50.0/24
      AvailabilityZone:
        Fn::Select:
          - 0
          - Fn::GetAZs: ""
      Tags:
      - Key: Name
        Value: !Sub ${AWS::StackName}-SecVpc-AZ1-TGW-Subnet

# SecVpc AZ2内创建TGW子网
  SecVpcAz2TgwSubnet:
    Type: AWS::EC2::Subnet
    Properties:
      VpcId: !Ref SecVpc
      CidrBlock: 10.100.60.0/24
      AvailabilityZone:
        Fn::Select:
          - 1
          - Fn::GetAZs: ""
      Tags:
      - Key: Name
        Value: !Sub ${AWS::StackName}-SecVpc-AZ2-TGW-Subnet

#---------------------------SecVpc创建路由表-------------------------------------#

# 公有子网路由表及关联
  SecVpcAz1PublicRouteTable:
    Type: "AWS::EC2::RouteTable"
    Properties:
      VpcId: !Ref SecVpc
      Tags:
        - Key: Name
          Value: !Sub ${AWS::StackName}-SecVpc-AZ1-Public-RouteTable

  SecVpcAz1PublicRouteTableAssociation:
    Type: "AWS::EC2::SubnetRouteTableAssociation"
    Properties:
      RouteTableId: !Ref SecVpcAz1PublicRouteTable
      SubnetId: !Ref SecVpcAz1PublicSubnet

  SecVpcAz2PublicRouteTable:
    Type: "AWS::EC2::RouteTable"
    Properties:
      VpcId: !Ref SecVpc
      Tags:
        - Key: Name
          Value: !Sub ${AWS::StackName}-SecVpc-AZ2-Public-RouteTable

  SecVpcAz2PublicRouteTableAssociation:
    Type: "AWS::EC2::SubnetRouteTableAssociation"
    Properties:
      RouteTableId: !Ref SecVpcAz2PublicRouteTable
      SubnetId: !Ref SecVpcAz2PublicSubnet

# Private子网路由表及关联
  SecVpcAz1PrivateRouteTable:
    Type: "AWS::EC2::RouteTable"
    Properties:
      VpcId: !Ref SecVpc
      Tags:
        - Key: Name
          Value: !Sub ${AWS::StackName}-SecVpc-AZ1-Private-RouteTable

  SecVpcAz1PrivateRouteTableAssociation:
    Type: "AWS::EC2::SubnetRouteTableAssociation"
    Properties:
      RouteTableId: !Ref SecVpcAz1PrivateRouteTable
      SubnetId: !Ref SecVpcAz1PrivateSubnet

  SecVpcAz2PrivateRouteTable:
    Type: "AWS::EC2::RouteTable"
    Properties:
      VpcId: !Ref SecVpc
      Tags:
        - Key: Name
          Value: !Sub ${AWS::StackName}-SecVpc-AZ2-Private-RouteTable

  SecVpcAz2PrivateRouteTableAssociation:
    Type: "AWS::EC2::SubnetRouteTableAssociation"
    Properties:
      RouteTableId: !Ref SecVpcAz2PrivateRouteTable
      SubnetId: !Ref SecVpcAz2PrivateSubnet


# Tgw路由表及关联
  SecVpcAz1TgwRouteTable:
    Type: "AWS::EC2::RouteTable"
    Properties:
      VpcId: !Ref SecVpc
      Tags:
        - Key: Name
          Value: !Sub ${AWS::StackName}-SecVpc-AZ1-Tgw-RouteTable

  SecVpcAz1TgwRouteTableAssociation:
    Type: "AWS::EC2::SubnetRouteTableAssociation"
    Properties:
      RouteTableId: !Ref SecVpcAz1TgwRouteTable
      SubnetId: !Ref SecVpcAz1TgwSubnet

  SecVpcAz2TgwRouteTable:
    Type: "AWS::EC2::RouteTable"
    Properties:
      VpcId: !Ref SecVpc
      Tags:
        - Key: Name
          Value: !Sub ${AWS::StackName}-SecVpc-AZ2-Tgw-RouteTable

  SecVpcAz2TgwRouteTableAssociation:
    Type: "AWS::EC2::SubnetRouteTableAssociation"
    Properties:
      RouteTableId: !Ref SecVpcAz2TgwRouteTable
      SubnetId: !Ref SecVpcAz2TgwSubnet

#---------------------------NAT Gateway------------------------------------#

# AZ1 NAT GW
  SecVpcAz1NatGatewayEIP:
     Type: AWS::EC2::EIP
     Properties:
        Tags:
        - Key: Name
          Value: !Sub ${AWS::StackName}-SecVpc-AZ1-NatGateway-EIP

  SecVpcAz1NatGateway:
     Type: AWS::EC2::NatGateway
     Properties:
        AllocationId: !GetAtt SecVpcAz1NatGatewayEIP.AllocationId
        SubnetId: !Ref SecVpcAz1PublicSubnet
        Tags:
        - Key: Name
          Value: !Sub ${AWS::StackName}-SecVpc-AZ1-NatGateway

# AZ2 NAT GW
  SecVpcAz2NatGatewayEIP:
     Type: AWS::EC2::EIP
     Properties:
        Tags:
        - Key: Name
          Value: !Sub ${AWS::StackName}-SecVpc-AZ2-NatGateway-EIP

  SecVpcAz2NatGateway:
     Type: AWS::EC2::NatGateway
     Properties:
        AllocationId: !GetAtt SecVpcAz2NatGatewayEIP.AllocationId
        SubnetId: !Ref SecVpcAz2PublicSubnet
        Tags:
        - Key: Name
          Value: !Sub ${AWS::StackName}-SecVpc-AZ2-NatGateway

  SecVpcAz1PrivateSubnetToInternetRoute:
     DependsOn: SecVpcAz1NatGateway
     Type: AWS::EC2::Route
     Properties:
        RouteTableId: !Ref SecVpcAz1PrivateRouteTable
        DestinationCidrBlock: '0.0.0.0/0'
        NatGatewayId: !Ref SecVpcAz1NatGateway

  SecVpcAz2PrivateSubnetToInternetRoute:
     DependsOn: SecVpcAz2NatGateway
     Type: AWS::EC2::Route
     Properties:
        RouteTableId: !Ref SecVpcAz2PrivateRouteTable
        DestinationCidrBlock: '0.0.0.0/0'
        NatGatewayId: !Ref SecVpcAz2NatGateway

#---------------------------添加路由------------------------------------#

# 公有子网添加默认路由去往IGW
  SecVpcAz1PublicSubnetToInternetRoute:
    Type: "AWS::EC2::Route"
    DependsOn: SecVpcIGW
    Properties:
     RouteTableId: !Ref SecVpcAz1PublicRouteTable
     DestinationCidrBlock: 0.0.0.0/0
     GatewayId: !Ref SecVpcIGW

  SecVpcAz2PublicSubnetToInternetRoute:
    Type: "AWS::EC2::Route"
    DependsOn: SecVpcIGW
    Properties:
     RouteTableId: !Ref SecVpcAz2PublicRouteTable
     DestinationCidrBlock: 0.0.0.0/0
     GatewayId: !Ref SecVpcIGW

#---------------------------SecVpc创建安全组------------------------------------#
  SecVpcSg:
    Type: AWS::EC2::SecurityGroup
    Properties:
      GroupDescription: SG to test ping
      VpcId: !Ref SecVpc
      SecurityGroupIngress:
      - IpProtocol: tcp
        FromPort: 22
        ToPort: 22
        CidrIp: 0.0.0.0/0
      - IpProtocol: icmp
        FromPort: -1
        ToPort: -1
        CidrIp: 0.0.0.0/0
      - IpProtocol: -1
        FromPort: -1
        ToPort: -1
        CidrIp: 10.100.0.0/16
      - IpProtocol: tcp
        FromPort: 3389
        ToPort: 3389
        CidrIp: 0.0.0.0/0
      Tags:
        - Key: Name
          Value: !Sub ${AWS::StackName}-SecVpcSg


#---------------------------SecVpc创建EC2实例------------------------------------#

#--------------------------IAM Instance Role and Profile------------------------------------#
  ApplianceRole:
    Type: AWS::IAM::Role
    Properties:
      RoleName: !Sub "${AWS::StackName}-appliance-role"
      ManagedPolicyArns:
        - "arn:aws:iam::aws:policy/AmazonSSMManagedInstanceCore"
      AssumeRolePolicyDocument:
        Statement:
          - Effect: Allow
            Principal:
              Service:
                - ec2.amazonaws.com
            Action:
              - 'sts:AssumeRole'
      Path: /

  AppliancePolicy:
    Type: AWS::IAM::Policy
    Properties:
      PolicyName: AppServer
      PolicyDocument:
        Statement:
          - Effect: Allow
            Action:
              - ec2:DescribeNetworkInterfaces
            Resource: '*'
      Roles:
        - !Ref ApplianceRole

  ApplianceProfile:
    Type: AWS::IAM::InstanceProfile
    Properties:
      Path: /
      Roles:
        - !Ref ApplianceRole

#---------------------------SecVpc创建GWLB------------------------------------#

# Gateway Load Balancer (GWLB), Target Group, Listener
  Gwlb:
    Type: AWS::ElasticLoadBalancingV2::LoadBalancer
    Properties:
      LoadBalancerAttributes:
        - Key: load_balancing.cross_zone.enabled
          Value: true
      Name: gwlb1
      Type: gateway
      Subnets:
        - !Ref SecVpcAz1PrivateSubnet
        - !Ref SecVpcAz2PrivateSubnet
      Tags:
      - Key: Name
        Value: !Sub "${AWS::StackName}-gwlb-1"

  # Target Group:
  TargetGroup:
    Type: AWS::ElasticLoadBalancingV2::TargetGroup
    Properties:
      Name: tg1
      Port: 6081
      Protocol: GENEVE
      TargetGroupAttributes:
      - Key: deregistration_delay.timeout_seconds
        Value: "20"
      VpcId: !Ref SecVpc
      HealthCheckPort: 80
      HealthCheckProtocol: HTTP
      TargetType: instance
      Targets:
        - Id: !Ref Appliance1
        - Id: !Ref Appliance2
      Tags:
      - Key: Name
        Value: !Sub "${AWS::StackName}-tg-1"

  # Listener:
  Listener:
    Type: AWS::ElasticLoadBalancingV2::Listener
    Properties:
      DefaultActions:
      - Type: forward
        TargetGroupArn: !Ref TargetGroup
      LoadBalancerArn: !Ref Gwlb

#---------------------------SecVpc创建EC2实例------------------------------------#


# EC2 Instances (Appliances acting as target for GWLB):
  Appliance1:
    DependsOn: [Gwlb, SecVpcAz1PrivateSubnetToInternetRoute]
    Type: AWS::EC2::Instance
    Properties:
      ImageId: !Ref EC2InstanceAmiId
      KeyName: !Ref MyKeyPair
      InstanceType: t2.micro
      IamInstanceProfile: !Ref ApplianceProfile
      SecurityGroupIds:
        - !Ref SecVpcSg
      SubnetId: !Ref SecVpcAz1PrivateSubnet
      Tags:
        - Key: Name
          Value: !Sub "${AWS::StackName}-appliance-1"
      UserData:
        Fn::Base64: |
          #!/bin/bash -ex

          # Install packages:
          yum update -y;
          yum install jq -y;
          yum install httpd -y;
          yum install htop -y;
          sudo yum install iptables-services -y;

          # Enable IP Forwarding:
          echo "net.ipv4.ip_forward = 1" >> /etc/sysctl.d/00-defaults.conf;
          sysctl -p /etc/sysctl.d/00-defaults.conf;

          # Configure hostname:
          hostnamectl set-hostname gwlb-target-1;

          # Configure SSH client alive interval for ssh session timeout:
          echo 'ClientAliveInterval 60' | sudo tee --append /etc/ssh/sshd_config;
          service sshd restart;

          # Set dark background for vim:
          touch /home/ec2-user/.vimrc;
          echo "set background=dark" >> /home/ec2-user/.vimrc;

          # Define variables:
          curl --silent http://169.254.169.254/latest/dynamic/instance-identity/document > /home/ec2-user/iid;
          export instance_interface=$(curl --silent http://169.254.169.254/latest/meta-data/network/interfaces/macs/);
          export instance_vpcid=$(curl --silent http://169.254.169.254/latest/meta-data/network/interfaces/macs/$instance_interface/vpc-id);
          export instance_az=$(cat /home/ec2-user/iid |grep 'availability' | awk -F': ' '{print $2}' | awk -F',' '{print $1}');
          export instance_ip=$(cat /home/ec2-user/iid |grep 'privateIp' | awk -F': ' '{print $2}' | awk -F',' '{print $1}' | awk -F'"' '{print$2}');
          export instance_region=$(cat /home/ec2-user/iid |grep 'region' | awk -F': ' '{print $2}' | awk -F',' '{print $1}' | awk -F'"' '{print$2}');
          export local_az_gwlb_ip=$(aws --region $instance_region ec2 describe-network-interfaces --filters Name=vpc-id,Values=$instance_vpcid | jq ' .NetworkInterfaces[] | select(.AvailabilityZone=='$instance_az') | select(.InterfaceType=="gateway_load_balancer") |.PrivateIpAddress' -r);
          export remote_az_gwlb_ip=$(aws --region $instance_region ec2 describe-network-interfaces --filters Name=vpc-id,Values=$instance_vpcid | jq ' .NetworkInterfaces[] | select(.AvailabilityZone!='$instance_az') | select(.InterfaceType=="gateway_load_balancer") |.PrivateIpAddress' -r);


          # Start http and configure index.html:
          systemctl enable httpd;
          systemctl start httpd;
          touch /var/www/html/index.html;
          echo "<html>" >> /var/www/html/index.html
          echo "  <head>" >> /var/www/html/index.html
          echo "    <title>Gateway Load Balancer POC</title>" >> /var/www/html/index.html
          echo "    <meta http-equiv='Content-Type' content='text/html; charset=ISO-8859-1'>" >> /var/www/html/index.html
          echo "  </head>" >> /var/www/html/index.html
          echo "  <body>" >> /var/www/html/index.html
          echo "    <h1>Welcome to Gateway Load Balancer POC:</h1>" >> /var/www/html/index.html
          echo "    <h2>This is appliance running in $instance_az. Happy testing!</h2>" >> /var/www/html/index.html
          echo "  </body>" >> /var/www/html/index.html
          echo "</html>" >> /var/www/html/index.html

          # Start and configure iptables:
          systemctl enable iptables;
          systemctl start iptables;

          # Configuration below allows allows all traffic:
          # Set the default policies for each of the built-in chains to ACCEPT:
          iptables -P INPUT ACCEPT;
          iptables -P FORWARD ACCEPT;
          iptables -P OUTPUT ACCEPT;

          # Flush the nat and mangle tables, flush all chains (-F), and delete all non-default chains (-X):
          iptables -t nat -F;
          iptables -t mangle -F;
          iptables -F;
          iptables -X;

          # Configure nat table to hairpin traffic back to GWLB:
          iptables -t nat -A PREROUTING -p udp -s $local_az_gwlb_ip -d $instance_ip -i eth0 -j DNAT --to-destination $local_az_gwlb_ip:6081;
          iptables -t nat -A POSTROUTING -p udp --dport 6081 -s $local_az_gwlb_ip -d $local_az_gwlb_ip -o eth0 -j MASQUERADE;

          iptables -t nat -A PREROUTING -p udp -s $remote_az_gwlb_ip -d $instance_ip -i eth0 -j DNAT --to-destination $remote_az_gwlb_ip:6081;
          iptables -t nat -A POSTROUTING -p udp --dport 6081 -s $remote_az_gwlb_ip -d $remote_az_gwlb_ip -o eth0 -j MASQUERADE;

          # Save iptables:
          service iptables save;

  Appliance2:
    DependsOn: [ Gwlb, SecVpcAz2PrivateSubnetToInternetRoute ]
    Type: AWS::EC2::Instance
    Properties:
      ImageId: !Ref EC2InstanceAmiId
      KeyName: !Ref MyKeyPair
      InstanceType: t2.micro
      IamInstanceProfile: !Ref ApplianceProfile
      SecurityGroupIds:
        - !Ref SecVpcSg
      SubnetId: !Ref SecVpcAz2PrivateSubnet
      Tags:
        - Key: Name
          Value: !Sub "${AWS::StackName}-appliance-2"
      UserData:
        Fn::Base64: |
          #!/bin/bash -ex

          # Install packages:
          yum update -y;
          yum install jq -y;
          yum install httpd -y;
          yum install htop -y;
          sudo yum install iptables-services -y;

          # Enable IP Forwarding:
          echo "net.ipv4.ip_forward = 1" >> /etc/sysctl.d/00-defaults.conf;
          sysctl -p /etc/sysctl.d/00-defaults.conf;

          # Configure hostname:
          hostnamectl set-hostname gwlb-target-2;

          # Configure SSH client alive interval for ssh session timeout:
          echo 'ClientAliveInterval 60' | sudo tee --append /etc/ssh/sshd_config;
          service sshd restart;

          # Set dark background for vim:
          touch /home/ec2-user/.vimrc;
          echo "set background=dark" >> /home/ec2-user/.vimrc;

          # Define variables:
          curl --silent http://169.254.169.254/latest/dynamic/instance-identity/document > /home/ec2-user/iid;
          export instance_interface=$(curl --silent http://169.254.169.254/latest/meta-data/network/interfaces/macs/);
          export instance_vpcid=$(curl --silent http://169.254.169.254/latest/meta-data/network/interfaces/macs/$instance_interface/vpc-id);
          export instance_az=$(cat /home/ec2-user/iid |grep 'availability' | awk -F': ' '{print $2}' | awk -F',' '{print $1}');
          export instance_ip=$(cat /home/ec2-user/iid |grep 'privateIp' | awk -F': ' '{print $2}' | awk -F',' '{print $1}' | awk -F'"' '{print$2}');
          export instance_region=$(cat /home/ec2-user/iid |grep 'region' | awk -F': ' '{print $2}' | awk -F',' '{print $1}' | awk -F'"' '{print$2}');
          export local_az_gwlb_ip=$(aws --region $instance_region ec2 describe-network-interfaces --filters Name=vpc-id,Values=$instance_vpcid | jq ' .NetworkInterfaces[] | select(.AvailabilityZone=='$instance_az') | select(.InterfaceType=="gateway_load_balancer") |.PrivateIpAddress' -r);
          export remote_az_gwlb_ip=$(aws --region $instance_region ec2 describe-network-interfaces --filters Name=vpc-id,Values=$instance_vpcid | jq ' .NetworkInterfaces[] | select(.AvailabilityZone!='$instance_az') | select(.InterfaceType=="gateway_load_balancer") |.PrivateIpAddress' -r);


          # Start http and configure index.html:
          systemctl enable httpd;
          systemctl start httpd;
          touch /var/www/html/index.html;
          echo "<html>" >> /var/www/html/index.html
          echo "  <head>" >> /var/www/html/index.html
          echo "    <title>Gateway Load Balancer POC</title>" >> /var/www/html/index.html
          echo "    <meta http-equiv='Content-Type' content='text/html; charset=ISO-8859-1'>" >> /var/www/html/index.html
          echo "  </head>" >> /var/www/html/index.html
          echo "  <body>" >> /var/www/html/index.html
          echo "    <h1>Welcome to Gateway Load Balancer POC:</h1>" >> /var/www/html/index.html
          echo "    <h2>This is appliance running in $instance_az. Happy testing!</h2>" >> /var/www/html/index.html
          echo "  </body>" >> /var/www/html/index.html
          echo "</html>" >> /var/www/html/index.html

          # Start and configure iptables:
          systemctl enable iptables;
          systemctl start iptables;

          # Configuration below allows allows all traffic:
          # Set the default policies for each of the built-in chains to ACCEPT:
          iptables -P INPUT ACCEPT;
          iptables -P FORWARD ACCEPT;
          iptables -P OUTPUT ACCEPT;

          # Flush the nat and mangle tables, flush all chains (-F), and delete all non-default chains (-X):
          iptables -t nat -F;
          iptables -t mangle -F;
          iptables -F;
          iptables -X;

          # Configure nat table to hairpin traffic back to GWLB:
          iptables -t nat -A PREROUTING -p udp -s $local_az_gwlb_ip -d $instance_ip -i eth0 -j DNAT --to-destination $local_az_gwlb_ip:6081;
          iptables -t nat -A POSTROUTING -p udp --dport 6081 -s $local_az_gwlb_ip -d $local_az_gwlb_ip -o eth0 -j MASQUERADE;

          iptables -t nat -A PREROUTING -p udp -s $remote_az_gwlb_ip -d $instance_ip -i eth0 -j DNAT --to-destination $remote_az_gwlb_ip:6081;
          iptables -t nat -A POSTROUTING -p udp --dport 6081 -s $remote_az_gwlb_ip -d $remote_az_gwlb_ip -o eth0 -j MASQUERADE;

          # Save iptables:
          service iptables save;


  SecVpcBastionLinux:
    Type: AWS::EC2::Instance
    Properties:
      IamInstanceProfile: !Ref BastionSsmProfile
      ImageId: !Ref EC2InstanceAmiId
      KeyName: !Ref MyKeyPair
      InstanceType: t3.nano
      NetworkInterfaces:
        - AssociatePublicIpAddress: true
          DeviceIndex: 0
          GroupSet:
            - Ref: SecVpcSg
          SubnetId: !Ref SecVpcAz1PublicSubnet
      Tags:
        - Key: Name
          Value: !Sub ${AWS::StackName}-SecVpc-Bastion-Linux

# Create VPC Endpoint Service:

  VpcEndpointService:
    Type: AWS::EC2::VPCEndpointService
    Properties:
      GatewayLoadBalancerArns:
        - !Ref Gwlb
      AcceptanceRequired: false

# Create Lambda Custom Resource to retrieve VPC Endpoint Service Name:

  VpceServiceLambdaExecutionRole:
    Type: AWS::IAM::Role
    Properties:
      AssumeRolePolicyDocument:
        Version: 2012-10-17
        Statement:
          - Effect: Allow
            Principal:
              Service:
                - lambda.amazonaws.com
            Action:
              - sts:AssumeRole
      Path: /
      Policies:
        - PolicyName: root
          PolicyDocument:
            Version: 2012-10-17
            Statement:
              - Effect: Allow
                Action:
                  - logs:CreateLogStream
                  - logs:PutLogEvents
                Resource: arn:aws:logs:*:*:*
              - Effect: Allow
                Action:
                  - ec2:DescribeVpcEndpointServiceConfigurations
                  - ec2:DescribeVpcEndpointServicePermissions
                  - ec2:DescribeVpcEndpointServices
                Resource: "*"


  VpceServiceLogGroup:
    Type: AWS::Logs::LogGroup
    Properties:
        LogGroupName: !Sub /aws/lambda/${AWS::StackName}-service
        RetentionInDays: 1

  VpceServiceName:
    Type: AWS::Lambda::Function
    DependsOn: VpceServiceLogGroup
    Properties:
      FunctionName: !Sub ${AWS::StackName}-service
      Handler: "index.handler"
      Role: !GetAtt VpceServiceLambdaExecutionRole.Arn
      Code:
        ZipFile: |
          import json
          import logging
          import time

          import boto3
          import cfnresponse
          from botocore.exceptions import ClientError

          try:
              ec2 = boto3.client('ec2')
          except ClientError as e:
              logger.error(f"ERROR: failed to connect to EC2 client: {e}")
              sys.exit(1)

          def handler(event, context):
              logger = logging.getLogger()
              logger.setLevel(logging.INFO)
              logger.info('Received event: {}'.format(json.dumps(event)))

              responseData = {}
              responseStatus = cfnresponse.FAILED

              try:
                  serviceid = event["ResourceProperties"]["VpceServiceId"]
              except Exception as e:
                  logger.info('Attribute retrival failure: {}'.format(e))

              try:
                  if event["RequestType"] == "Delete":
                      responseStatus = cfnresponse.SUCCESS
                      cfnresponse.send(event, context, responseStatus, responseData)
              except Exception:
                  logger.exception("Signaling failure to CloudFormation.")
                  cfnresponse.send(event, context, cfnresponse.FAILED, {})

              if event["RequestType"] == "Create":
                  logger.info("Retrieving VPC Endpoint Service Name:")
                  try:
                      response = ec2.describe_vpc_endpoint_service_configurations(
                          Filters=[
                              {
                                  'Name': 'service-id',
                                  'Values': [serviceid]
                              }
                          ]
                      )
                  except Exception as e:
                      logger.info('ec2.describe_vpc_endpoint_service_configurations failure: {}'.format(e))

                  service_name = response['ServiceConfigurations'][0]['ServiceName']

                  time.sleep(120)

                  responseData['ServiceName'] = service_name
                  responseStatus = cfnresponse.SUCCESS
                  cfnresponse.send(event, context, responseStatus, responseData)
      Runtime: python3.7
      Timeout: 150

  RetrieveVpceServiceName:
    Type: Custom::RetrieveAttributes
    Properties:
      ServiceToken: !GetAtt VpceServiceName.Arn
      VpceServiceId: !Ref VpcEndpointService

#=========================================VpcB========================================#
# 创建VpcB
  VpcB:
    Type: AWS::EC2::VPC
    Properties:
      CidrBlock: 10.120.10.0/16
      EnableDnsSupport: 'true'
      EnableDnsHostnames: 'true'
      Tags:
       - Key: Name
         Value: !Sub ${AWS::StackName}-VpcB

# 创建IGW并且关联到VPC
  VpcBIGW:
    Type: "AWS::EC2::InternetGateway"
    Properties:
      Tags:
        - Key: Name
          Value: !Sub ${AWS::StackName}-VpcBIGW

  VpcBAttachIgw:
    Type: "AWS::EC2::VPCGatewayAttachment"
    Properties:
      VpcId: !Ref VpcB
      InternetGatewayId: !Ref VpcBIGW

#---------------------------VpcB创建8个子网-------------------------------------#

# VpcB AZ1内创建公有子网
  VpcBAz1PublicSubnet:
    Type: AWS::EC2::Subnet
    Properties:
      VpcId: !Ref VpcB
      CidrBlock: 10.120.10.0/24
      AvailabilityZone:
        Fn::Select:
          - 0
          - Fn::GetAZs: ""
      Tags:
      - Key: Name
        Value: !Sub ${AWS::StackName}-VpcB-AZ1-Public-Subnet

# VpcB AZ2内创建公有子网
  VpcBAz2PublicSubnet:
    Type: AWS::EC2::Subnet
    Properties:
      VpcId: !Ref VpcB
      CidrBlock: 10.120.20.0/24
      AvailabilityZone:
        Fn::Select:
          - 1
          - Fn::GetAZs: ""
      Tags:
      - Key: Name
        Value: !Sub ${AWS::StackName}-VpcB-AZ2-Public-Subnet

# VpcB AZ1内创建APP子网
  VpcBAz1AppSubnet:
    Type: AWS::EC2::Subnet
    Properties:
      VpcId: !Ref VpcB
      CidrBlock: 10.120.30.0/24
      AvailabilityZone:
        Fn::Select:
          - 0
          - Fn::GetAZs: ""
      Tags:
      - Key: Name
        Value: !Sub ${AWS::StackName}-VpcB-AZ1-App-Subnet

# VpcB AZ2内创建APP子网
  VpcBAz2AppSubnet:
    Type: AWS::EC2::Subnet
    Properties:
      VpcId: !Ref VpcB
      CidrBlock: 10.120.40.0/24
      AvailabilityZone:
        Fn::Select:
          - 1
          - Fn::GetAZs: ""
      Tags:
      - Key: Name
        Value: !Sub ${AWS::StackName}-VpcB-AZ2-App-Subnet

# VpcB AZ1内创建GWLB子网
  VpcBAz1GwlbeSubnet:
    Type: AWS::EC2::Subnet
    Properties:
      VpcId: !Ref VpcB
      CidrBlock: 10.120.50.0/24
      AvailabilityZone:
        Fn::Select:
          - 0
          - Fn::GetAZs: ""
      Tags:
      - Key: Name
        Value: !Sub ${AWS::StackName}-VpcB-AZ1-GWLBe-Subnet

# VpcB AZ2内创建GWLB子网
  VpcBAz2GwlbeSubnet:
    Type: AWS::EC2::Subnet
    Properties:
      VpcId: !Ref VpcB
      CidrBlock: 10.120.60.0/24
      AvailabilityZone:
        Fn::Select:
          - 1
          - Fn::GetAZs: ""
      Tags:
      - Key: Name
        Value: !Sub ${AWS::StackName}-VpcB-AZ2-GWLBe-Subnet

# VpcB AZ1内创建TGW子网
  VpcBAz1TgwSubnet:
    Type: AWS::EC2::Subnet
    Properties:
      VpcId: !Ref VpcB
      CidrBlock: 10.120.70.0/24
      AvailabilityZone:
        Fn::Select:
          - 0
          - Fn::GetAZs: ""
      Tags:
      - Key: Name
        Value: !Sub ${AWS::StackName}-VpcB-AZ1-TGW-Subnet

# VpcB AZ2内创建TGW子网
  VpcBAz2TgwSubnet:
    Type: AWS::EC2::Subnet
    Properties:
      VpcId: !Ref VpcB
      CidrBlock: 10.120.80.0/24
      AvailabilityZone:
        Fn::Select:
          - 1
          - Fn::GetAZs: ""
      Tags:
      - Key: Name
        Value: !Sub ${AWS::StackName}-VpcB-AZ2-TGW-Subnet

#---------------------------VpcB创建路由表-------------------------------------#

# 公有子网路由表及关联
  VpcBAz1PublicRouteTable:
    Type: "AWS::EC2::RouteTable"
    Properties:
      VpcId: !Ref VpcB
      Tags:
        - Key: Name
          Value: !Sub ${AWS::StackName}-VpcB-AZ1-Public-RouteTable

  VpcBAz1PublicRouteTableAssociation:
    Type: "AWS::EC2::SubnetRouteTableAssociation"
    Properties:
      RouteTableId: !Ref VpcBAz1PublicRouteTable
      SubnetId: !Ref VpcBAz1PublicSubnet

  VpcBAz2PublicRouteTable:
    Type: "AWS::EC2::RouteTable"
    Properties:
      VpcId: !Ref VpcB
      Tags:
        - Key: Name
          Value: !Sub ${AWS::StackName}-VpcB-AZ2-Public-RouteTable

  VpcBAz2PublicRouteTableAssociation:
    Type: "AWS::EC2::SubnetRouteTableAssociation"
    Properties:
      RouteTableId: !Ref VpcBAz2PublicRouteTable
      SubnetId: !Ref VpcBAz2PublicSubnet

# App子网路由表及关联
  VpcBAz1AppRouteTable:
    Type: "AWS::EC2::RouteTable"
    Properties:
      VpcId: !Ref VpcB
      Tags:
        - Key: Name
          Value: !Sub ${AWS::StackName}-VpcB-AZ1-App-RouteTable

  VpcBAz1AppRouteTableAssociation:
    Type: "AWS::EC2::SubnetRouteTableAssociation"
    Properties:
      RouteTableId: !Ref VpcBAz1AppRouteTable
      SubnetId: !Ref VpcBAz1AppSubnet

  VpcBAz2AppRouteTable:
    Type: "AWS::EC2::RouteTable"
    Properties:
      VpcId: !Ref VpcB
      Tags:
        - Key: Name
          Value: !Sub ${AWS::StackName}-VpcB-AZ2-App-RouteTable

  VpcBAz2AppRouteTableAssociation:
    Type: "AWS::EC2::SubnetRouteTableAssociation"
    Properties:
      RouteTableId: !Ref VpcBAz2AppRouteTable
      SubnetId: !Ref VpcBAz2AppSubnet

# GWLBe子网路由表及关联
  VpcBAz1GwlbeRouteTable:
    Type: "AWS::EC2::RouteTable"
    Properties:
      VpcId: !Ref VpcB
      Tags:
        - Key: Name
          Value: !Sub ${AWS::StackName}-VpcB-AZ1-GWLBe-RouteTable

  VpcBAz1GwlbeRouteTableAssociation:
    Type: "AWS::EC2::SubnetRouteTableAssociation"
    Properties:
      RouteTableId: !Ref VpcBAz1GwlbeRouteTable
      SubnetId: !Ref VpcBAz1GwlbeSubnet

  VpcBAz2GwlbeRouteTable:
    Type: "AWS::EC2::RouteTable"
    Properties:
      VpcId: !Ref VpcB
      Tags:
        - Key: Name
          Value: !Sub ${AWS::StackName}-VpcB-AZ2-GWLBe-RouteTable

  VpcBAz2GwlbeRouteTableAssociation:
    Type: "AWS::EC2::SubnetRouteTableAssociation"
    Properties:
      RouteTableId: !Ref VpcBAz2GwlbeRouteTable
      SubnetId: !Ref VpcBAz2GwlbeSubnet

# Tgw路由表及关联
  VpcBAz1TgwRouteTable:
    Type: "AWS::EC2::RouteTable"
    Properties:
      VpcId: !Ref VpcB
      Tags:
        - Key: Name
          Value: !Sub ${AWS::StackName}-VpcB-AZ1-Tgw-RouteTable

  VpcBAz1TgwRouteTableAssociation:
    Type: "AWS::EC2::SubnetRouteTableAssociation"
    Properties:
      RouteTableId: !Ref VpcBAz1TgwRouteTable
      SubnetId: !Ref VpcBAz1TgwSubnet

  VpcBAz2TgwRouteTable:
    Type: "AWS::EC2::RouteTable"
    Properties:
      VpcId: !Ref VpcB
      Tags:
        - Key: Name
          Value: !Sub ${AWS::StackName}-VpcB-AZ2-Tgw-RouteTable

  VpcBAz2TgwRouteTableAssociation:
    Type: "AWS::EC2::SubnetRouteTableAssociation"
    Properties:
      RouteTableId: !Ref VpcBAz2TgwRouteTable
      SubnetId: !Ref VpcBAz2TgwSubnet

#---------------------------NAT Gateway------------------------------------#

# AZ1 NAT GW
  VpcBAz1NatGatewayEIP:
     Type: AWS::EC2::EIP
     Properties:
        Tags:
        - Key: Name
          Value: !Sub ${AWS::StackName}-VpcB-AZ1-NatGateway-EIP

  VpcBAz1NatGateway:
     Type: AWS::EC2::NatGateway
     Properties:
        AllocationId: !GetAtt VpcBAz1NatGatewayEIP.AllocationId
        SubnetId: !Ref VpcBAz1PublicSubnet
        Tags:
        - Key: Name
          Value: !Sub ${AWS::StackName}-VpcB-AZ1-NatGateway

# AZ2 NAT GW
  VpcBAz2NatGatewayEIP:
     Type: AWS::EC2::EIP
     Properties:
        Tags:
        - Key: Name
          Value: !Sub ${AWS::StackName}-VpcB-AZ2-NatGateway-EIP

  VpcBAz2NatGateway:
     Type: AWS::EC2::NatGateway
     Properties:
        AllocationId: !GetAtt VpcBAz2NatGatewayEIP.AllocationId
        SubnetId: !Ref VpcBAz2PublicSubnet
        Tags:
        - Key: Name
          Value: !Sub ${AWS::StackName}-VpcB-AZ2-NatGateway

# VpcB 创建Endpoint
  VpcBEndpoint1:
    Type: AWS::EC2::VPCEndpoint
    Properties:
      VpcId: !Ref VpcB
      ServiceName: !GetAtt RetrieveVpceServiceName.ServiceName
      VpcEndpointType: GatewayLoadBalancer
      SubnetIds:
        - !Ref VpcBAz1GwlbeSubnet

  VpcBEndpoint2:
    Type: AWS::EC2::VPCEndpoint
    Properties:
      VpcId: !Ref VpcB
      ServiceName: !GetAtt RetrieveVpceServiceName.ServiceName
      VpcEndpointType: GatewayLoadBalancer
      SubnetIds:
        - !Ref VpcBAz2GwlbeSubnet

#---------------------------添加路由------------------------------------#

# 公有子网添加默认路由去往IGW
  VpcBAz1PublicSubnetToInternetRoute:
    Type: "AWS::EC2::Route"
    DependsOn: VpcBIGW
    Properties:
     RouteTableId: !Ref VpcBAz1PublicRouteTable
     DestinationCidrBlock: 0.0.0.0/0
     GatewayId: !Ref VpcBIGW

  VpcBAz2PublicSubnetToInternetRoute:
    Type: "AWS::EC2::Route"
    DependsOn: VpcBIGW
    Properties:
     RouteTableId: !Ref VpcBAz2PublicRouteTable
     DestinationCidrBlock: 0.0.0.0/0
     GatewayId: !Ref VpcBIGW

# App子网添加默认路由去往GWLBe
  VpcBAz1AppSubnetRoute1:
    Type: "AWS::EC2::Route"
    DependsOn: VpcBEndpoint2
    Properties:
     RouteTableId: !Ref VpcBAz1AppRouteTable
     DestinationCidrBlock: 10.120.20.0/24
     VpcEndpointId: !Ref VpcBEndpoint2

  VpcBAz1AppSubnetRoute2:
    Type: "AWS::EC2::Route"
    DependsOn: VpcBEndpoint1
    Properties:
     RouteTableId: !Ref VpcBAz1AppRouteTable
     DestinationCidrBlock: 0.0.0.0/0
     VpcEndpointId: !Ref VpcBEndpoint1

  VpcBAz2AppSubnetRoute1:
    Type: "AWS::EC2::Route"
    DependsOn: VpcBEndpoint1
    Properties:
     RouteTableId: !Ref VpcBAz2AppRouteTable
     DestinationCidrBlock: 10.120.10.0/24
     VpcEndpointId: !Ref VpcBEndpoint1

  VpcBAz2AppSubnetRoute2:
    Type: "AWS::EC2::Route"
    DependsOn: VpcBEndpoint2
    Properties:
     RouteTableId: !Ref VpcBAz2AppRouteTable
     DestinationCidrBlock: 0.0.0.0/0
     VpcEndpointId: !Ref VpcBEndpoint2


# Gwlbe子网添加默认路由去往NAT GW
  VpcBAz1GwlbeSubnetRoute1:
    Type: "AWS::EC2::Route"
    DependsOn: VpcBAz1NatGateway
    Properties:
     RouteTableId: !Ref VpcBAz1GwlbeRouteTable
     DestinationCidrBlock: 0.0.0.0/0
     NatGatewayId: !Ref VpcBAz1NatGateway

  VpcBAz2GwlbeSubnetRoute1:
    Type: "AWS::EC2::Route"
    DependsOn: VpcBAz2NatGateway
    Properties:
     RouteTableId: !Ref VpcBAz2GwlbeRouteTable
     DestinationCidrBlock: 0.0.0.0/0
     NatGatewayId: !Ref VpcBAz2NatGateway

#---------------------------VpcB创建安全组------------------------------------#
  VpcBBastionVpcSg:
    Type: AWS::EC2::SecurityGroup
    Properties:
      GroupDescription: SG to test ping
      VpcId: !Ref VpcB
      SecurityGroupIngress:
      - IpProtocol: tcp
        FromPort: 22
        ToPort: 22
        CidrIp: 0.0.0.0/0
      - IpProtocol: icmp
        FromPort: -1
        ToPort: -1
        CidrIp: 0.0.0.0/0
      - IpProtocol: -1
        FromPort: -1
        ToPort: -1
        CidrIp: 10.120.0.0/16
      - IpProtocol: tcp
        FromPort: 3389
        ToPort: 3389
        CidrIp: 0.0.0.0/0
      - IpProtocol: tcp
        FromPort: !Ref WebServerPort
        ToPort: !Ref WebServerPort
        CidrIp: 0.0.0.0/0
      Tags:
        - Key: Name
          Value: !Sub ${AWS::StackName}-VpcBSg


#---------------------------VpcB创建EC2实例------------------------------------#

  VpcBBastionLinux:
    Type: AWS::EC2::Instance
    Properties:
      IamInstanceProfile: !Ref BastionSsmProfile
      ImageId: !Ref EC2InstanceAmiId
      KeyName: !Ref MyKeyPair
      InstanceType: t3.nano
      NetworkInterfaces:
        - AssociatePublicIpAddress: true
          DeviceIndex: 0
          GroupSet:
            - Ref: VpcBBastionVpcSg
          SubnetId: !Ref VpcBAz1PublicSubnet
      Tags:
        - Key: Name
          Value: !Sub ${AWS::StackName}-VpcB-Bastion-Linux

  VpcBApp1Linux:
    Type: AWS::EC2::Instance
    DependsOn: [ VpcBEndpoint1, VpcBAz2AppSubnetRoute1 ]
    Properties:
      IamInstanceProfile: !Ref BastionSsmProfile
      ImageId: !Ref EC2InstanceAmiId
      KeyName: !Ref MyKeyPair
      InstanceType: t3.nano
      NetworkInterfaces:
        - DeviceIndex: 0
          GroupSet:
            - Ref: VpcBBastionVpcSg
          SubnetId: !Ref VpcBAz1AppSubnet
      UserData:
        Fn::Base64:
          !Sub |
          #!/bin/bash
          yum update -y
          yum install -y httpd
          sed -i.bak 's/Listen 80/Listen ${WebServerPort}/g' /etc/httpd/conf/httpd.conf
          echo "<h2>Hello World from $(hostname -f)</h2>" > /var/www/html/index.html
          systemctl start httpd.service
          systemctl enable httpd.service
      Tags:
        - Key: Name
          Value: !Sub ${AWS::StackName}-VpcB-App1-Linux

  VpcBApp2Linux:
    Type: AWS::EC2::Instance
    DependsOn: [VpcBEndpoint2, VpcBAz2AppSubnetRoute2]
    Properties:
      IamInstanceProfile: !Ref BastionSsmProfile
      ImageId: !Ref EC2InstanceAmiId
      KeyName: !Ref MyKeyPair
      InstanceType: t3.nano
      NetworkInterfaces:
        - DeviceIndex: 0
          GroupSet:
            - Ref: VpcBBastionVpcSg
          SubnetId: !Ref VpcBAz2AppSubnet
      UserData:
        Fn::Base64:
          !Sub |
          #!/bin/bash
          yum update -y
          yum install -y httpd
          sed -i.bak 's/Listen 80/Listen ${WebServerPort}/g' /etc/httpd/conf/httpd.conf
          echo "<h2>Hello World from $(hostname -f)</h2>" > /var/www/html/index.html
          systemctl start httpd.service
          systemctl enable httpd.service
      Tags:
        - Key: Name
          Value: !Sub ${AWS::StackName}-VpcB-App2-Linux

# -----------创建ALB-----------
  VpcBEc2TargetGroup:
    Type: AWS::ElasticLoadBalancingV2::TargetGroup
    Properties:
      HealthCheckIntervalSeconds: 30
      HealthCheckProtocol: HTTP
      HealthCheckTimeoutSeconds: 15
      HealthyThresholdCount: 5
      Matcher:
        HttpCode: '200'
      Name: VpcBEc2TargetGroup
      Port: !Ref WebServerPort
      Protocol: HTTP
      TargetGroupAttributes:
      - Key: deregistration_delay.timeout_seconds
        Value: '20'
      Targets:
      - Id: !Ref VpcBApp1Linux
        Port: !Ref WebServerPort
      - Id: !Ref VpcBApp2Linux
        Port: !Ref WebServerPort
      UnhealthyThresholdCount: 3
      VpcId: !Ref VpcB
      Tags:
        - Key: Name
          Value: VpcB-Target-Group

  VpcBAlbListener:
    Type: AWS::ElasticLoadBalancingV2::Listener
    Properties:
      DefaultActions:
        - Type: forward
          TargetGroupArn: !Ref VpcBEc2TargetGroup
      LoadBalancerArn: !Ref VpcBApplicationLoadBalancer
      Port: !Ref WebServerPort
      Protocol: HTTP

  VpcBApplicationLoadBalancer:
    DependsOn: VpcBIGW
    Type: AWS::ElasticLoadBalancingV2::LoadBalancer
    Properties:
      Name: !Sub ${AWS::StackName}-VpcB-ALB
      Type: application
      Scheme: internet-facing
      Subnets:
      - !Ref VpcBAz1PublicSubnet
      - !Ref VpcBAz2PublicSubnet
      SecurityGroups:
      - !GetAtt VpcBBastionVpcSg.GroupId

# -----------创建WAF-----------

  WafTestPermitCnIpWebACL:
    Type: 'AWS::WAFv2::WebACL'
    Properties:
      Name: waf-test-web-acl
      Scope: REGIONAL
      Description: WebACL permit all from china ip address.
      DefaultAction:
        Block: {}
      VisibilityConfig:
        SampledRequestsEnabled: true
        CloudWatchMetricsEnabled: true
        MetricName: permit-cn
      Rules:
        - Name: waf-test-permit-cn-ipaddress
          Priority: 0
          Statement:
            GeoMatchStatement:
              CountryCodes:
              - CN
          Action:
            Allow: {}
          VisibilityConfig:
            SampledRequestsEnabled: true
            CloudWatchMetricsEnabled: true
            MetricName: permit-cn
      Tags:
        - Key: Name
          Value: waf-test-permit-cn-webacl

  WafTestPermitCnIpWebACLAssociation:
    Type: 'AWS::WAFv2::WebACLAssociation'
    Properties:
      WebACLArn: !GetAtt WafTestPermitCnIpWebACL.Arn
      ResourceArn: !Ref VpcBApplicationLoadBalancer

# -------------自定义资源修改路由表------------------------
# Create Lambda Custom Resource to edit existing route in an RTB
  ReplaceRouteLambdaExecutionRole:
    Type: AWS::IAM::Role
    Properties:
      AssumeRolePolicyDocument:
        Version: 2012-10-17
        Statement:
          - Effect: Allow
            Principal:
              Service:
                - lambda.amazonaws.com
            Action:
              - sts:AssumeRole
      Path: /
      Policies:
        - PolicyName: root
          PolicyDocument:
            Version: 2012-10-17
            Statement:
              - Effect: Allow
                Action:
                  - logs:CreateLogStream
                  - logs:PutLogEvents
                Resource: arn:aws:logs:*:*:*
              - Effect: Allow
                Action:
                  - ec2:ReplaceRoute
                Resource: "*"

  ReplaceRouteLogGroup:
    Type: AWS::Logs::LogGroup
    Properties:
        LogGroupName: !Sub /aws/lambda/${AWS::StackName}ReplaceRoute
        RetentionInDays: 1

  ReplaceRoute:
    Type: AWS::Lambda::Function
    DependsOn: ReplaceRouteLogGroup
    Properties:
      FunctionName: !Sub ${AWS::StackName}ReplaceRoute
      Handler: "index.handler"
      Role: !GetAtt ReplaceRouteLambdaExecutionRole.Arn
      Code:
        ZipFile: |
          import json
          import logging
          import time
          import boto3
          import cfnresponse
          from botocore.exceptions import ClientError
          try:
              ec2 = boto3.client('ec2')
          except ClientError as e:
              logger.error(f"ERROR: failed to connect to EC2 client: {e}")
              sys.exit(1)
          def handler(event, context):
              logger = logging.getLogger()
              logger.setLevel(logging.INFO)
              logger.info('Received event: {}'.format(json.dumps(event)))
              responseData = {}
              responseStatus = cfnresponse.FAILED
              try:
                  DestCidr = event["ResourceProperties"]["DestCidr"]
                  VpceId = event["ResourceProperties"]["VpceId"]
                  RtbId = event["ResourceProperties"]["RtbId"]

              except Exception as e:
                  logger.info('Attribute retrival failure: {}'.format(e))
              try:
                  if event["RequestType"] == "Delete":
                      response = ec2.replace_route(
                          LocalTarget = True,
                          DestinationCidrBlock = DestCidr,
                          RouteTableId = RtbId
                      )
                      responseStatus = cfnresponse.SUCCESS
                      cfnresponse.send(event, context, responseStatus, responseData)
              except Exception:
                  logger.exception("Signaling failure to CloudFormation.")
                  cfnresponse.send(event, context, cfnresponse.FAILED, {})
              if event["RequestType"] == "Create":
                  logger.info(f"Replacing target to {VpceId} for {DestCidr} for {RtbId}")
                  try:
                      response = ec2.replace_route(
                          DestinationCidrBlock = DestCidr,
                          VpcEndpointId = VpceId,
                          RouteTableId = RtbId
                      )
                  except Exception as e:
                      logger.info('ec2.describe_vpc_endpoint_service_configurations failure: {}'.format(e))

                  responseStatus = cfnresponse.SUCCESS
                  cfnresponse.send(event, context, responseStatus, responseData)
      Runtime: python3.7
      Timeout: 150

# 修改路由表
  VpcBAz1PublicRtb1EditLocalRoute:
    Type: Custom::ReplaceRoute
    DependsOn: VpcBEndpoint1
    Properties:
      ServiceToken: !GetAtt ReplaceRoute.Arn
      DestCidr: 10.120.0.0/16
      VpceId: !Ref VpcBEndpoint1
      RtbId: !Ref VpcBAz1PublicRouteTable

  VpcBAz2PublicRtb1EditLocalRoute:
    Type: Custom::ReplaceRoute
    DependsOn: VpcBEndpoint2
    Properties:
      ServiceToken: !GetAtt ReplaceRoute.Arn
      DestCidr: 10.120.0.0/16
      VpceId: !Ref VpcBEndpoint2
      RtbId: !Ref VpcBAz2PublicRouteTable

  VpcBAz1AppRtb1EditLocalRoute:
    Type: Custom::ReplaceRoute
    DependsOn: VpcBEndpoint1
    Properties:
      ServiceToken: !GetAtt ReplaceRoute.Arn
      DestCidr: 10.120.0.0/16
      VpceId: !Ref VpcBEndpoint1
      RtbId: !Ref VpcBAz1AppRouteTable

  VpcBAz2AppRtb1EditLocalRoute:
    Type: Custom::ReplaceRoute
    DependsOn: VpcBEndpoint2
    Properties:
      ServiceToken: !GetAtt ReplaceRoute.Arn
      DestCidr: 10.120.0.0/16
      VpceId: !Ref VpcBEndpoint2
      RtbId: !Ref VpcBAz1AppRouteTable

三、测试

3.1 访问ALB测试

复制ALB的DNS名称。

访问ALB的DNS名称测试业务连通性。

3.2 APP抓包测试

连接App2

通过Session Manger连接。

在App上抓取TCP 8443的报文，ALB会终结HTTP会话，所以在APP上看到的是来自ALB的请求地址。

另外可以通过curl ifconfig.me查看App访问公网的IP地址，是NAT GW2的公网IP地址。

sudo -i
tcpdump -n port 8443

3.3 appliance抓包测试

在App1上ping公网地址。

在appliance上抓取GENEVE的报文，并使用grep对ICMP报文进行过滤。

root@gwlb-target-1 ~]# tcpdump -n -i eth0 port 6081 | grep ICMP

3.4 防火墙高可用测试

在App1上发起对公网地址的访问，同时在appliance1和appliance2上开启抓包，确认流量目前通过appliance1转发。

Stop正在转发流量的appliance实例。

等待大概1分钟左右（依据是否为新建会话，还有协议切换时间不同），流量切换到另外一台防火墙。因为GWLB开启了Cross-zone load balancing，所以流量可以跨AZ，通过appliance2转发流量。

3.5 AWS WAF 策略测试

查看创建的Web ACLs。

规则只允许来自中国区的IP地址，来自其他区域的IP请求会被拒绝。

查看策略详情。

开启代理服务器之后，从日本发起对ALB的访问请求，访问会被拒绝。

查看Web ACLs的日志记录，来自日本的请求被拒绝JP，来自中国区的流量允许访问。