Deployment using CloudFormation

AWS CloudFormation is a service that provides a common language for describing and provisioning all the infrastructure resources in your cloud environment. CloudFormation allows you to use a simple text file to model and provision all the resources needed for their applications across all regions and accounts in an automated and secure manner. This file is known as a CloudFormation template, and it defines the resources and their properties.

Traceable fetches the cloudwatch logs every 5 minutes, parses the data, and sends it to the Traceable platform. The processed data is displayed in the Traceable UI. The following flow diagram shows how the traffic flows through:

The document details the steps for deploying the infrastructure via the AWS Management Console and AWS CLI, outlining the necessary parameters and configurations for a successful deployment.

Before you begin

Make a note of the following points before proceeding with configurations:

• Keep Traceable's access token handy. It will be used (optionally) when configuring the variables in the CloudFormation template. You can copy the access token by logging into your Traceable account and then navigating to Settings ()→Account → Agent Token.

• Download the template.yaml file from Traceable’s download site. Navigate to install → aws-api-gateway → cloudformation → latest.

REST API gateway

To monitor the REST API gateway, complete the following:

1. Navigate to the console. aws.amazon.com/apigateway/.

2. Select the API and then select the stage.

3. Navigate to Logs/tracing. 

4. Select Full Request and Response Logs from Cloudwatch settings → Cloudwatch Logs drop-down list.

5. Mark Enable Access Logging as true under Custom Access Logging and append the following JSON in Log Format:

   JSONJSON

   {
     "requestId":"$context.requestId","ip":"$context.identity.sourceIp","httpMethod":"$context.httpMethod","path":"$context.path","status":"$context.status","responseLength":"$context.responseLength","domainName":"$context.domainName"}

HTTP API gateway

To monitor the HTTP API gateway, enable access logging and append the JSON mentioned above in the Log Format.

Configure AWS

Configure AWS in your shell and verify that the region is set correctly. Enter the following command to set up your AWS CLI installation:

aws configure

The following example shows sample values. Replace them with your values to configure the credentials correctly.

$ aws configure
AWS Access Key ID [None]: AKIAIOSFODNN7EXAMPLE
AWS Secret Access Key [None]: wJalrXUtnFEMI/K7MDENG/bPxRfiCYEXAMPLEKEY
Default region name [None]: us-west-2
Default output format [None]: json

For more information on the credentials file, see Configuration and credential file settings.

If you have configured named AWS profiles, export the environment variable AWS_PROFILE=myprofile where the profile named myprofile has the credentials that you wish to use to deploy the Traceable mirroring resources.

Finally, run the following command and verify that the AWS region is set to the region where you wish to install Traceable:

aws configure get region

Alternatively, you can configure the following environment variables:

• AWS_ACCESS_KEY_ID

• AWS_SECRET_ACCESS_KEY

• AWS_REGION or AWS_DEFAULT_REGION

• AWS_DEFAULT_OUTPUT

For more configuration information, see AWS documentation.

Multiple Accounts

A single AWS API Gateway account can support multiple accounts across different regions. However, each region within an account must be configured separately. This setup requires establishing cross-account IAM roles in each account, allowing the host account to authenticate and access the necessary resources.

Deploy CloudFormation through the AWS console

Log into your AWS account and complete the following steps:

1. Search for CloudFormation in your AWS console and click CloudFormation.

   

2. Click on Create Stack.

   

3. In the Prerequisite—Prepare template section, Choose an existing template; in the Specify template section, select Upload a template file.

   

4. Specify a stack name and configure the various template file values.

   

5. Add tags, select rollback options, and configure additional settings as needed.

6. Review the settings and create the stack. Monitor the progress and ensure the stack is created successfully.

Deploy using AWS CLI

Complete the following steps to deploy the CloudFormation template using AWS CLI:

1. Configure AWS as detailed in the Configure AWS section.

2. Update the parameters as required in the following command. The tags in the command are optional and are applied to all instances of the created stack where tags are supported.

   aws cloudformation deploy \
     --template-file deployments/aws-api-gateway/cloudformation/template.yaml \
     --stack-name traceable-apigw-dev \
     --capabilities CAPABILITY_NAMED_IAM \
     --parameter-overrides Accounts=W3siYXBpX2xpc3QiOlsiNjI1dmV \
     CrossAccountRoleARNs=arn:aws:iam::<***>:role/<***>,arn:aws:iam::<***>:role/<***> \
     TraceableAgentEndpoint=1.2.3.4 \
     TraceableServiceName=production \
     InstanceType=m4.xlarge \
     SubnetId=subnet-0fbc2fd16 \
     VpcId=vpc-02726c2b6ef \
     --tags key1=value1 key2=value2 key3=value3

Detailed Explanation

1. aws cloudformation deploy:

   • This command is used to deploy a CloudFormation stack using the AWS CLI.

2. --template-file deployments/aws-api-gateway/cloudformation/template.yaml:

   • Specifies the file path to the CloudFormation template. This template file contains the definitions of the AWS resources to be created or updated.

3. --stack-name traceable-apigw-dev:

   • Sets the name of the CloudFormation stack to traceable-apigw-dev. The stack name must be unique within an AWS region.

4. --capabilities CAPABILITY_NAMED_IAM:

   • Acknowledges that the stack might create IAM resources with custom names. This capability is required when the template includes creating or modifying IAM roles, policies, or users.

5. --parameter-overrides:

   • Provides a list of parameters to override the default values defined in the CloudFormation template. Each parameter is specified as a Key=Value pair.

   Parameters in this example:

   • Accounts=W3siYXBpX2xpc3QiOlsiNjI1dmV:

     • This appears to be a complex parameter, possibly JSON or an encoded string, specifying account-related configurations.

   • CrossAccountRoleARNs=arn:aws:iam::<***>:role/<***>,arn:aws:iam::<***>:role/<***>:

     • Specifies ARNs (Amazon Resource Names) of IAM roles that allow the stack to assume roles across different accounts.

   • TraceableAgentEndpoint=1.2.3.4:

     • Defines the endpoint for the Traceable Agent, where the agent will report data.

   • TraceableServiceName=production:

     • Sets the service name for Traceable traffic as "production."

   • InstanceType=m4.xlarge:

     • Specifies the EC2 instance type to be used; in this case, m4.xlarge.

   • SubnetId=subnet-0fbc2fd16:

     • The ID of the subnet where the EC2 instance will be deployed.

   • VpcId=vpc-02726c2b6ef:

     • The ID of the VPC (Virtual Private Cloud) where the subnet exists.

6. --tags key1=value1 key2=value2 key3=value3:

   • Adds tags to the stack resources. Tags are key-value pairs used to categorize and organize AWS resources.

   • Example tags:

     • key1=value1

     • key2=value2

     • key3=value3

Note

• Replace placeholders (<***>) with actual values.

• Ensure that the template file path (--template-file) is correct and accessible from where the command is executed.

• Validate that the IAM roles, VPC, and subnet exist and are correctly referenced in your AWS environment.

Parameters

The following tables explain the various parameters used in the CloudFormation template.

AWS API Gateway tracing agent configuration

arn:aws:iam::4324:role/customer-cross-account-role,
arn:aws:iam::44324:role/test-cust-cross-account-role

Traceable Platform agent configuration

Configure these only when you wish to deploy the Traceable Platform Agent alongside the tracing agent instance. This configuration is applicable only when you have configured TraceableAgentEndpoint as http://localhost:5442.

Traceable EC2 instance configuration

The following table describes the Traceable EC2 instance configuration

Advanced configuration

Accounts

Accounts is a base-64 encoded JSON  array where each object is comprised of the following fields.

The following is an example JSON file:

[
  {
    "api_list": [
      "6v0f",
      "4323jnj"
    ],
    "cross_account_role_arn": "",
    "exclude": false,
    "region": "us-east-1"
  },
  {
    "api_list": [
      "example-api"
    ],
    "cross_account_role_arn": "",
    "exclude": false,
    "region": "us-east-2"
  },
  {
    "api_list": [
      "lw1lrm2"
    ],
    "cross_account_role_arn": "arn:aws:iam::4324:role/customer-cross-account-role",
    "exclude": true,
    "region": "us-east-1"
  },
  {
    "api_list": [
      "example-api"
    ],
    "cross_account_role_arn": "arn:aws:iam::4324:role/customer-cross-account-role",
    "exclude": true,
    "region": "us-east-2"
  },
  {
    "api_list": [],
    "cross_account_role_arn": "arn:aws:iam::44324:role/test-cust-cross-account-role",
    "exclude": true,
    "region": "us-east-2"
  }
]

Following is an explanation of the example JSON:

1. The first object is for the region us-east-1 in Account 1. It will capture traffic only from the API gateway instance with the ID 6v0f, 4323jnj in this account. It is assumed that Account 1 is the host account (where the agent instance is deployed).

2. The second object is for the region us-east-2. It will capture traffic only from the API gateway instance with the ID example-api in this account.

3. The third object is for the region us-east-1 in Account 2. It will capture traffic from all the API gateway instances in this region in account 2 except API gateway instances with IDlw1lrm2 . The cross_account_role_arn in this case is the ARN of the IAM Role authenticating Account 1 to access required objects in Account 2.

4. The fourth object is for the region us-east-2 in Account 2. It will capture traffic from all the API gateway instances in this region in account 2 except from API gateway instances with IDexample-api .The cross_account_role_arn in this case is the ARN of the IAM Role authenticating Account 1 to access required objects in Account 2.

   Note

   The IAM role ARN is the same as that of case number 3.

5. The fifth object is for the region us-east-2 in Account 3. It will capture traffic from all the API gateway instances in this region in this account. The cross_account_role_arn in this case is the ARN of the IAM Role authenticating Account 1 to access required objects in Account 3.

Following is a compact JSON. The compact JSON is base64 encoded and the string is passed as a value of Accounts parameter.

[{"api_list":["6v0f","4323jnj"],"cross_account_role_arn":"","exclude":false,"region":"us-east-1"},{"api_list":["lw1lrm2"],"cross_account_role_arn":"arn:aws:iam::4324:role/customer-cross-account-role","exclude":true,"region":"us-east-1"},{"api_list":[],"cross_account_role_arn":"arn:aws:iam::44324:role/test-cust-cross-account-role","exclude":true,"region":"us-east-2"}]

The following is the expected base64 encoded input of the example compact JSON mentioned above:

W3siYXBpX2xpc3QiOlsiNnYwZiIsIjQzMjNqbmoiXSwiY3Jvc3NfYWNjb3VudF9yb2xlX2FybiI6IiIsImV4Y2x1ZGUiOmZhbHNlLCJyZWdpb24iOiJ1cy1lYXN0LTEifSx7ImFwaV9saXN0IjpbImx3MWxybTIiXSwiY3Jvc3NfYWNjb3VudF9yb2xlX2FybiI6ImFybjphd3M6aWFtOjo0MzI0OnJvbGUvY3VzdG9tZXItY3Jvc3MtYWNjb3VudC1yb2xlIiwiZXhjbHVkZSI6dHJ1ZSwicmVnaW9uIjoidXMtZWFzdC0xIn0seyJhcGlfbGlzdCI6W10sImNyb3NzX2FjY291bnRfcm9sZV9hcm4iOiJhcm46YXdzOmlhbTo6NDQzMjQ6cm9sZS90ZXN0LWN1c3QtY3Jvc3MtYWNjb3VudC1yb2xlIiwiZ

Uninstall

You can uninstall the CloudFormation template through the AWS console or the CLI.

Uninstall through UI

Select the stack from the CloudFormation tab and click Delete to delete all the Traceable instances created to capture traffic on the AWS API gateway.

Uninstall using CLI

Enter the following command to delete all Traceable instances created to capture traffic on the AWS API gateway.

aws cloudformation delete-stack --stack-name <name of stack>