AWS API gateway monitoring

The Terraform template creates AWS resources to enable the monitoring of AWS API gateway logs. The template creates an EC2 instance where Traceable services are running. Traceable fetches the cloudwatch logs every 5 minutes, parses the data, and sends it to the Traceable platform. The processed data is displayed in the Traceable UI. The following flow diagram displays how the traffic flows through:

Before you begin

Make a note of the following points before proceeding with configurations:

• Keep Traceable's access token handy. It will be used when you configure the variables in the tfvars (*.tfvars) file. You can copy the access token by logging into your Traceable account and then navigating to Administration ()→Account → Agent Token.
• Make sure that Terraform is already installed. For more information on installing Terraform, see Download Terraform.
• REST API gateway- To monitor the REST API gateway, complete the following:
  1. Navigate to the console. aws.amazon.com/apigateway/.
  2. Select the API and then select the stage.
  3. Navigate to Logs/tracing. 
  4. Select Full Request and Response Logs from Cloudwatch settings → Cloudwatch Logs drop-down list.
  5. Mark Enable Access Logging as true under Custom Access Logging and append the following JSON in Log Format:
     JSONJSON

     {
       "requestId":"$context.requestId",
       "ip":"$context.identity.sourceIp",
       "httpMethod":"$context.httpMethod",
       "path":"$context.path",
       "status":"$context.status",
       "responseLength":"$context.responseLength",
       "domainName":"$context.domainName"
     }

• HTTP API gateway - To monitor the HTTP API gateway, enable access logging and append the JSON mentioned above in the Log Format.
• Configure AWS - Configure AWS in your shell and verify that the region is set correctly. Enter the following command to set up your AWS CLI installation:
  ActionScriptActionScript

  aws configure

  The following example shows sample values. Replace them with your values to configure the credentials correctly.
  ActionScriptActionScript

  $ aws configure
  AWS Access Key ID [None]: AKIAIOSFODNN7EXAMPLE
  AWS Secret Access Key [None]: wJalrXUtnFEMI/K7MDENG/bPxRfiCYEXAMPLEKEY
  Default region name [None]: us-west-2
  Default output format [None]: json

  For more information on the credentials file, see Configuration and credential file settings.

  If you have configured named AWS profiles, export the environment variable AWS_PROFILE=myprofile where the profile named myprofile has the credentials that you wish to use in deploying the Traceable mirroring resources.

  Finally, run the following command and verify that the AWS region is set to the region where you wish to install Traceable:

  aws configure get region

  For more information on configuration, see AWS documentation.

Download

Traceable provides AWS API gateway traffic mirroring tarball. Complete the following steps to download and untar the tarball: 

1. Enter the following command to download the tarball:
   ActionScriptActionScript

   curl -O https://downloads.traceable.ai/install/aws-api-gateway/terraform/latest/aws-api-gateway-tf.tar.gz

2. Untar the tarball. Enter the following command:
   ActionScriptActionScript

   tar xvzf aws-api-gateway-tf.tar.gz

3. Change directory. Enter the following command:
   ActionScriptActionScript

   cd aws-api-gateway-tf/

Create tfvars file

Create a terraform.tfvars file with terraform variables as shown below:

api_gateways = {
  action = "include"
  value  = ["a78dyas7al/preprod", "fgd77s90sa"]
}
subnet_id               = "subnet-078f70a209b0f4fa6"
key_name                = "my-key-pair"
traceable_refresh_token = "my-token"
traceable_environment   = "my-env"
traceable_service_name  = "my-service-name"
tags = {
  "TAG1"        = "VALUE1"
  "TAG2:GLOBAL" = "VALUE2"
}

Configuration variables

The following tables describe the various terraform variables.

api_gateways

The api_gateways object has the following values:

Example

The following example explains the configuration. The configuration shown below will monitor preprod stage of the API gateway with ID = a78dyas7al. It will also monitor all the stages of the API gateway with ID = fgd77s90sa.

api_gateways = {
  action = "include"
  value  = ["a78dyas7al/preprod", "fgd77s90sa"]
}

Apply terraform

As a last step to configure, run the following command to apply the terraform changes:

terraform init
terraform apply

Verification

Log into your Traceable Platform account and navigate to API Catalog → Services to view the service name (traceable_service_name) that you configured earlier.

Uninstall

To uninstall, run the following command from the same directory to destroy all the resources created in the installation step:

terraform destroy

Troubleshooting

Loss of traffic

If your setup is experiencing a loss of traffic or observing less amount of traffic, you may try doing the following:

1. Login to the Traceable EC2 instance that you created using the Terraform template earlier. The instance's name should be with the prefix traceable-instance-.
   Note

   Make sure to update the security group attached to this instance to allow SSH connection from your machine.
2. After you log in, check the service status of the running service. Enter the following command:
   ActionScriptActionScript

   sudo systemctl status traceable logstash

You may also view the logs generated by Traceable by navigating to the logs' directory:

cd /var/traceable/log/aws-api-gateway/
less gateway.err