- 09 Jul 2024
- 10 Minutes to read
- Print
- PDF
Deployment using CloudFormation
- Updated on 09 Jul 2024
- 10 Minutes to read
- Print
- PDF
AWS CloudFormation is a service that provides a common language for describing and provisioning all the infrastructure resources in your cloud environment. CloudFormation allows you to use a simple text file to model and provision all the resources needed for their applications across all regions and accounts in an automated and secure manner. This file is known as a CloudFormation template, and it defines the resources and their properties.
Traceable fetches the cloudwatch logs every 5 minutes, parses the data, and sends it to the Traceable platform. The processed data is displayed in the Traceable UI. The following flow diagram shows how the traffic flows through:
![](http://cdn.document360.io/24f14f07-13d1-4684-8fae-6d8f811768ee/Images/Documentation/traceable_aws_monitoring_traffic_flow.png)
The document details the steps for deploying the infrastructure via the AWS Management Console and AWS CLI, outlining the necessary parameters and configurations for a successful deployment.
Before you begin
Make a note of the following points before proceeding with configurations:
Keep Traceable's access token handy. It will be used (optionally) when configuring the variables in the CloudFormation template. You can copy the access token by logging into your Traceable account and then navigating to Settings (
)→Account → Agent Token.
Download the
template.yaml
file from Traceable’s download site. Navigate to install → aws-api-gateway → cloudformation → latest.
REST API gateway
To monitor the REST API gateway, complete the following:
Navigate to the console. aws.amazon.com/apigateway/.
Select the API and then select the stage.
Navigate to Logs/tracing.
Select Full Request and Response Logs from Cloudwatch settings → Cloudwatch Logs drop-down list.
Mark Enable Access Logging as
true
under Custom Access Logging and append the following JSON in Log Format:JSON
{ "requestId":"$context.requestId","ip":"$context.identity.sourceIp","httpMethod":"$context.httpMethod","path":"$context.path","status":"$context.status","responseLength":"$context.responseLength","domainName":"$context.domainName"}
HTTP API gateway
To monitor the HTTP API gateway, enable access logging and append the JSON mentioned above in the Log Format.
Configure AWS
Configure AWS in your shell and verify that the region is set correctly. Enter the following command to set up your AWS CLI installation:
ActionScript
aws configure
The following example shows sample values. Replace them with your values to configure the credentials correctly.
ActionScript
$ aws configure
AWS Access Key ID [None]: AKIAIOSFODNN7EXAMPLE
AWS Secret Access Key [None]: wJalrXUtnFEMI/K7MDENG/bPxRfiCYEXAMPLEKEY
Default region name [None]: us-west-2
Default output format [None]: json
For more information on the credentials
file, see Configuration and credential file settings.
If you have configured named AWS profiles, export the environment variable AWS_PROFILE=myprofile
where the profile named myprofile
has the credentials that you wish to use to deploy the Traceable mirroring resources.
Finally, run the following command and verify that the AWS region is set to the region where you wish to install Traceable:
aws configure get region
Alternatively, you can configure the following environment variables:
AWS_ACCESS_KEY_ID
AWS_SECRET_ACCESS_KEY
AWS_REGION or AWS_DEFAULT_REGION
AWS_DEFAULT_OUTPUT
For more configuration information, see AWS documentation.
Multiple Accounts
A single AWS API Gateway account can support multiple accounts across different regions. However, each region within an account must be configured separately. This setup requires establishing cross-account IAM roles in each account, allowing the host account to authenticate and access the necessary resources.
Deploy CloudFormation through the AWS console
Log into your AWS account and complete the following steps:
Search for CloudFormation in your AWS console and click CloudFormation.
Click on Create Stack.
In the Prerequisite—Prepare template section, Choose an existing template; in the Specify template section, select Upload a template file.
Specify a stack name and configure the various template file values.
Add tags, select rollback options, and configure additional settings as needed.
Review the settings and create the stack. Monitor the progress and ensure the stack is created successfully.
Deploy using AWS CLI
Complete the following steps to deploy the CloudFormation template using AWS CLI:
Configure AWS as detailed in the Configure AWS section.
Update the parameters as required in the following command. The tags in the command are optional and are applied to all instances of the created stack where tags are supported.
aws cloudformation deploy \ --template-file deployments/aws-api-gateway/cloudformation/template.yaml \ --stack-name traceable-apigw-dev \ --capabilities CAPABILITY_NAMED_IAM \ --parameter-overrides Accounts=W3siYXBpX2xpc3QiOlsiNjI1dmV \ CrossAccountRoleARNs=arn:aws:iam::<***>:role/<***>,arn:aws:iam::<***>:role/<***> \ TraceableAgentEndpoint=1.2.3.4 \ TraceableServiceName=production \ InstanceType=m4.xlarge \ SubnetId=subnet-0fbc2fd16 \ VpcId=vpc-02726c2b6ef \ --tags key1=value1 key2=value2 key3=value3
Detailed Explanation
aws cloudformation deploy
:This command is used to deploy a CloudFormation stack using the AWS CLI.
--template-file deployments/aws-api-gateway/cloudformation/template.yaml
:Specifies the file path to the CloudFormation template. This template file contains the definitions of the AWS resources to be created or updated.
--stack-name traceable-apigw-dev
:Sets the name of the CloudFormation stack to
traceable-apigw-dev
. The stack name must be unique within an AWS region.
--capabilities CAPABILITY_NAMED_IAM
:Acknowledges that the stack might create IAM resources with custom names. This capability is required when the template includes creating or modifying IAM roles, policies, or users.
--parameter-overrides
:Provides a list of parameters to override the default values defined in the CloudFormation template. Each parameter is specified as a
Key=Value
pair.
Parameters in this example:
Accounts=W3siYXBpX2xpc3QiOlsiNjI1dmV
:This appears to be a complex parameter, possibly JSON or an encoded string, specifying account-related configurations.
CrossAccountRoleARNs=arn:aws:iam::<***>:role/<***>,arn:aws:iam::<***>:role/<***>
:Specifies ARNs (Amazon Resource Names) of IAM roles that allow the stack to assume roles across different accounts.
TraceableAgentEndpoint=1.2.3.4
:Defines the endpoint for the Traceable Agent, where the agent will report data.
TraceableServiceName=production
:Sets the service name for Traceable traffic as "production."
InstanceType=m4.xlarge
:Specifies the EC2 instance type to be used; in this case,
m4.xlarge
.
SubnetId=subnet-0fbc2fd16
:The ID of the subnet where the EC2 instance will be deployed.
VpcId=vpc-02726c2b6ef
:The ID of the VPC (Virtual Private Cloud) where the subnet exists.
--tags key1=value1 key2=value2 key3=value3
:Adds tags to the stack resources. Tags are key-value pairs used to categorize and organize AWS resources.
Example tags:
key1=value1
key2=value2
key3=value3
Note
Replace placeholders (
<***>
) with actual values.Ensure that the template file path (
--template-file
) is correct and accessible from where the command is executed.Validate that the IAM roles, VPC, and subnet exist and are correctly referenced in your AWS environment.
Parameters
The following tables explain the various parameters used in the CloudFormation template.
AWS API Gateway tracing agent configuration
Parameter | Optional/Mandatory | Description | Default value |
---|---|---|---|
| Mandatory | Refer to Accounts section. | - |
| Mandatory | This parameter is a comma-separated list of ARNs. In the
| - |
| Mandatory | Enter a comma-separated list of ARNs of log groups for creating the IAM policy. For Example, | - |
| Optional | These will be the stack-wide tags. | - |
| Mandatory | Enter a comma-separated list of ARNs of stages of API Gateways for creating the IAM policy. For Example, | - |
| Optional | The Traceable Platform agent reporting endpoint for the agent. If not overridden, platform agent deployment will also be made on the same VM. For more information, see the following section. |
|
| Optional | The Traceable service name for the captured traffic. |
|
Traceable Platform agent configuration
Configure these only when you wish to deploy the Traceable Platform Agent alongside the tracing agent instance. This configuration is applicable only when you have configured TraceableAgentEndpoint
as http://localhost:5442
.
Parameter | Optional/Mandatory | Description | Default value |
---|---|---|---|
| Mandatory | Traceable Platform token. This is the token you generated as part of a step in the Before you begin section. | - |
| Optional | Defines the environment, such as dev, QE, staging, etc. |
|
| Optional | Traceable Platform API endpoint. |
|
| Optional | Traceable Platform agent version. | - |
Traceable EC2 instance configuration
The following table describes the Traceable EC2 instance configuration
Parameter | Optional/Mandatory | Description | Default value |
---|---|---|---|
| Optional | The type of instance for deploying the agent. | m4.xlarge |
| Mandatory | The subnet ID of the subnet where the agent instance should be deployed. | - |
| Mandatory | The VPC ID of the VPC in which the subnet given in | - |
| Mandatory | The SSH keyname attached to the Traceable EC2 instance. (By default, port number 22 is not opened by EC2 instance). | - |
| Optional | Set it to |
|
Advanced configuration
Parameter | Optional/Mandatory | Description | Default value |
---|---|---|---|
| Optional | Root directory path for traceable files. |
|
| Optional | Log directory path for traceable log files. |
|
| Optional | The maximum number of spans that you wish to bundle in a single request to the Traceable Platform agent. | 500 |
| Mandatory | AMI ID that is used to launch an instance. Set this variable when you want to use AMI ID to launch an instance. This takes priority over | - |
| Optional | Name of the Amazon Machine Image (AMI) to use (resolved from SSM). If the |
|
| Optional | ARN of the custom KMS Key to use to encrypt the EBS volume. |
|
| Optional | Storage capacity (in GB) to allocate. | 50 GB |
| Optional | AWS EBS volume type. For more information, see Amazon EBS volume types. | gb2 |
| Optional | Set it to |
|
| Optional | Device type for storage. |
|
| Optional | Traceable Platform agent version | - |
Accounts
Accounts is a base-64 encoded JSON array where each object is comprised of the following fields.
Field | Optional/Mandatory | Description |
---|---|---|
| Mandatory | The AWS region from where you wish to capture the API traffic. |
| Mandatory | When set to |
| Mandatory | List of API IDs to consider for the above action.
|
| Mandatory | The ARN of the IAM role that authenticates this account for accessing the required resources from another account. Provide an empty string if the listed API gateways are in the same account. |
The following is an example JSON file:
[
{
"api_list": [
"6v0f",
"4323jnj"
],
"cross_account_role_arn": "",
"exclude": false,
"region": "us-east-1"
},
{
"api_list": [
"example-api"
],
"cross_account_role_arn": "",
"exclude": false,
"region": "us-east-2"
},
{
"api_list": [
"lw1lrm2"
],
"cross_account_role_arn": "arn:aws:iam::4324:role/customer-cross-account-role",
"exclude": true,
"region": "us-east-1"
},
{
"api_list": [
"example-api"
],
"cross_account_role_arn": "arn:aws:iam::4324:role/customer-cross-account-role",
"exclude": true,
"region": "us-east-2"
},
{
"api_list": [],
"cross_account_role_arn": "arn:aws:iam::44324:role/test-cust-cross-account-role",
"exclude": true,
"region": "us-east-2"
}
]
Following is an explanation of the example JSON:
The first object is for the region
us-east-1
in Account 1. It will capture traffic only from the API gateway instance with the ID6v0f, 4323jnj
in this account. It is assumed that Account 1 is the host account (where the agent instance is deployed).The second object is for the region
us-east-2
. It will capture traffic only from the API gateway instance with the IDexample-api
in this account.The third object is for the region
us-east-1
in Account 2. It will capture traffic from all the API gateway instances in this region in account 2 except API gateway instances with IDlw1lrm2
. Thecross_account_role_arn
in this case is the ARN of the IAM Role authenticating Account 1 to access required objects in Account 2.The fourth object is for the region
us-east-2
in Account 2. It will capture traffic from all the API gateway instances in this region in account 2 except from API gateway instances with IDexample-api
.Thecross_account_role_arn
in this case is the ARN of the IAM Role authenticating Account 1 to access required objects in Account 2.Note
The IAM role ARN is the same as that of case number 3.
The fifth object is for the region
us-east-2
in Account 3. It will capture traffic from all the API gateway instances in this region in this account. Thecross_account_role_arn
in this case is the ARN of the IAM Role authenticating Account 1 to access required objects in Account 3.
Following is a compact JSON. The compact JSON is base64 encoded and the string is passed as a value of Accounts
parameter.
[{"api_list":["6v0f","4323jnj"],"cross_account_role_arn":"","exclude":false,"region":"us-east-1"},{"api_list":["lw1lrm2"],"cross_account_role_arn":"arn:aws:iam::4324:role/customer-cross-account-role","exclude":true,"region":"us-east-1"},{"api_list":[],"cross_account_role_arn":"arn:aws:iam::44324:role/test-cust-cross-account-role","exclude":true,"region":"us-east-2"}]
The following is the expected base64 encoded input of the example compact JSON mentioned above:
W3siYXBpX2xpc3QiOlsiNnYwZiIsIjQzMjNqbmoiXSwiY3Jvc3NfYWNjb3VudF9yb2xlX2FybiI6IiIsImV4Y2x1ZGUiOmZhbHNlLCJyZWdpb24iOiJ1cy1lYXN0LTEifSx7ImFwaV9saXN0IjpbImx3MWxybTIiXSwiY3Jvc3NfYWNjb3VudF9yb2xlX2FybiI6ImFybjphd3M6aWFtOjo0MzI0OnJvbGUvY3VzdG9tZXItY3Jvc3MtYWNjb3VudC1yb2xlIiwiZXhjbHVkZSI6dHJ1ZSwicmVnaW9uIjoidXMtZWFzdC0xIn0seyJhcGlfbGlzdCI6W10sImNyb3NzX2FjY291bnRfcm9sZV9hcm4iOiJhcm46YXdzOmlhbTo6NDQzMjQ6cm9sZS90ZXN0LWN1c3QtY3Jvc3MtYWNjb3VudC1yb2xlIiwiZ
Uninstall
You can uninstall the CloudFormation template through the AWS console or the CLI.
Uninstall through UI
Select the stack from the CloudFormation tab and click Delete to delete all the Traceable instances created to capture traffic on the AWS API gateway.
![traceable_aws_apigw_delete_stack_UI.png](http://cdn.document360.io/24f14f07-13d1-4684-8fae-6d8f811768ee/Images/Documentation/traceable_aws_apigw_delete_stack_UI.png)
Uninstall using CLI
Enter the following command to delete all Traceable instances created to capture traffic on the AWS API gateway.
aws cloudformation delete-stack --stack-name <name of stack>