AWS API gateway monitoring
  • 22 Dec 2023
  • 5 Minutes to read
  • PDF

AWS API gateway monitoring

  • PDF

Article Summary

The Terraform template creates AWS resources to enable the monitoring of AWS API gateway logs. The template creates an EC2 instance where Traceable services are running. Traceable fetches the cloudwatch logs every 5 minutes, parses the data, and sends it to the Traceable platform. The processed data is displayed in the Traceable UI. The following flow diagram displays how the traffic flows through:

AWS API Gateway currently limits log events to 1024 bytes. Log events larger than 1024 bytes, such as request and response headers/bodies, are truncated by API Gateway before submission to CloudWatch Logs. For information on important notes about AWS API gateway, see AWS docs.

Before you begin

Make a note of the following points before proceeding with configurations:

  • Keep Traceable's access token handy. It will be used when you configure the variables in the tfvars (*.tfvars) file. You can copy the access token by logging into your Traceable account and then navigating to Administration (image-1638268402925)AccountAgent Token.
  • Make sure that Terraform is already installed. For more information on installing Terraform, see Download Terraform.
  • REST API gateway- To monitor the REST API gateway, complete the following:
    1. Navigate to the console.
    2. Select the API and then select the stage.
    3. Navigate to Logs/tracing. 
    4. Select Full Request and Response Logs from Cloudwatch settings → Cloudwatch Logs drop-down list.
    5. Mark Enable Access Logging as true under Custom Access Logging and append the following JSON in Log Format:
  • HTTP API gateway - To monitor the HTTP API gateway, enable access logging and append the JSON mentioned above in the Log Format.
  • Configure AWS - Configure AWS in your shell and verify that the region is set correctly. Enter the following command to set up your AWS CLI installation:
    aws configure
    The following example shows sample values. Replace them with your values to configure the credentials correctly.
    $ aws configure
    AWS Secret Access Key [None]: wJalrXUtnFEMI/K7MDENG/bPxRfiCYEXAMPLEKEY
    Default region name [None]: us-west-2
    Default output format [None]: json
    For more information on the credentials file, see Configuration and credential file settings.

    If you have configured named AWS profiles, export the environment variable AWS_PROFILE=myprofile where the profile named myprofile has the credentials that you wish to use in deploying the Traceable mirroring resources.

    Finally, run the following command and verify that the AWS region is set to the region where you wish to install Traceable:

    aws configure get region

    For more information on configuration, see AWS documentation.


Traceable provides AWS API gateway traffic mirroring tarball. Complete the following steps to download and untar the tarball: 

  1. Enter the following command to download the tarball:
    curl -O
  2. Untar the tarball. Enter the following command:
    tar xvzf aws-api-gateway-tf.tar.gz
  3. Change directory. Enter the following command:
    cd aws-api-gateway-tf/

Create tfvars file

Create a terraform.tfvars file with terraform variables as shown below:

api_gateways = {
  action = "include"
  value  = ["a78dyas7al/preprod", "fgd77s90sa"]
subnet_id               = "subnet-078f70a209b0f4fa6"
key_name                = "my-key-pair"
traceable_refresh_token = "my-token"
traceable_environment   = "my-env"
traceable_service_name  = "my-service-name"
tags = {
  "TAG1"        = "VALUE1"

Configuration variables

The following tables describe the various terraform variables.

NameTypeDefault valueDescription
api_gatewaysobject-Refer to the next api_gateways section for the description.
subnet_idstring""The subnet ID where the Traceable instance is created.


string""SSH keyname attached to the Traceable EC2 instance.
instance_typestringm4.xlargeThe type of Traceable instance.


string""Traceable Platform token. This is the token that you have generated as part of a step in the Before you begin section.
traceable_environmentstringtraffic-mirroringEnvironment under which the mirrored traffic will be seen in the Traceable UI.
traceable_service_namestringtraffic-mirroring-serviceThe service name of the mirrored traffic. You will see this name on the API Catalog → Services UI.


stringapi.traceable.aiTraceable API Endpoint.


map(string){}Additional tags to be applied to all the Traceable resources. This is an optional variable.


The api_gateways object has the following values:

NameTypeDefault valueDescription
actionstringexcludeConfigures the action to be taken on the valuelist. The possible values are:
  • include - Configure the value as include when you wish to monitor all API gateways listed in value.
  • exclude - Configure the value as exclude when you wish to monitor all API gateways except the ones listed in value.
valuelist(string)[]List of API IDs to consider for the above action
  • API_ID - If action = include, then monitor the API gateway that has ID = API_ID. However, if action = exclude, then do not monitor this API gateway.
  • API_ID/STAGE_NAME - If action = include, then monitor the API gateway stage with name = STAGE_NAME and ID = API_ID. If action = exclude, then do not monitor this stage.
By default, the above variables are configured to monitor all the API gateways.


The following example explains the configuration. The configuration shown below will monitor preprod stage of the API gateway with ID = a78dyas7al. It will also monitor all the stages of the API gateway with ID = fgd77s90sa.

api_gateways = {
  action = "include"
  value  = ["a78dyas7al/preprod", "fgd77s90sa"]

Apply terraform

As a last step to configure, run the following command to apply the terraform changes:

terraform init
terraform apply


Log into your Traceable Platform account and navigate to API Catalog → Services to view the service name (traceable_service_name) that you configured earlier.


To uninstall, run the following command from the same directory to destroy all the resources created in the installation step:

terraform destroy


Loss of traffic

If your setup is experiencing a loss of traffic or observing less amount of traffic, you may try doing the following:

  1. Login to the Traceable EC2 instance that you created using the Terraform template earlier. The instance's name should be with the prefix traceable-instance-.
    Make sure to update the security group attached to this instance to allow SSH connection from your machine.
  2. After you log in, check the service status of the running service. Enter the following command:
    sudo systemctl status traceable logstash

You may also view the logs generated by Traceable by navigating to the logs' directory:

cd /var/traceable/log/aws-api-gateway/
less gateway.err

Was this article helpful?