Apigee - MessageLogging Policy

Apigee is a platform for developing and managing APIs. By fronting services with a proxy layer, Apigee provides an abstraction for your backend service APIs, including security, rate limiting, quotas, analytics, and more. Traceable offers a message for capturing requests and responses passing through your Apigee deployment to detect anomalous behavior or attacks. The following deployment diagram shows a high-level traffic flow architecture through Apigee and Traceable agent. Traceable's policy captures the incoming request and makes an asynchronous call to Traceable's platform.

Note that the Apigee agent sends data to the Platform agent through a TCP syslog call.

Before you begin

Make a note of the following points before you start with the deployment:

• The document expects a basic working knowledge of the Apigee environment.

• Traceable supports Apigee-x, hybrid, and Apigee Edge cloud.

• Ensure the Traceable platform agent is already deployed on a VM accessible from the Apigee environment. For more information, see Virtual Machine.

  Important

  Traceable’s MessageLogging Apigee agent requires a minimum Traceable Platform agent version 1.43.0.

• Note the IP address of the Traceable platform agent. This will be used in the Traceable policy configuration.

• Download the Traceable’s MessageLogging Policy

• Ensure that port 8444 (or the configured port for the Apigee log server) of the instance where the Platform agent is installed is accessible from the Apigee servers.

  Note

  Apigee log server is a part of Traceable Platform agent. The port number of Apigee log server is part of Traceable Platform agent configuration. The Apigee log server recieves data from Apigee.

• Note that the port of the Apigee log server is a TCP port. Make sure that from your Apigee setup, there should be outgoing data access to the TCP port of the Apigee log server (part of the Platform agent).

Download

Download the Traceable policy for Apigee from Traceable's download site. Navigate to agent → apigee → latest to download the traceable-shareflow.zip file. Save the file. This file would be used in the Configuration section.

Configuration

The configuration of the Traceable policy for Apigee consists of the following two steps:

1. Adding Traceable agent for Apigee as sharedflow

2. Attaching Traceable agent to:

   1. All API proxies using FlowHooks, or

   2. To a specific API proxy only

Policy bundle

The policy bundle has the following two files and their function:

• ExportSpansML.xml — This file has the Traceable Platform agent endpoint and TLS configuration.

  <?xml version="1.0" encoding="UTF-8" standalone="yes"?>
  <MessageLogging continueOnError="true" enabled="true" name="ExportSpansML">
    <Syslog>
      <Message>{TRACEABLE_DATA}__SPAN_END__</Message>
      <!-- Host IP/address TPA -->
      <Host></Host>
      <Port>8444</Port>
      <Protocol>TCP</Protocol>
      <PayloadOnly>true</PayloadOnly>
      <SSLInfo>
        <Enabled>false</Enabled>
        <ClientAuthEnabled>false</ClientAuthEnabled>
        <TrustStore>ref://give_your_truststore_reference</TrustStore>
        <IgnoreValidationErrors>false</IgnoreValidationErrors>
      </SSLInfo>
    </Syslog>
    <logLevel>ALERT</logLevel>
  </MessageLogging4

• TraceableJS.xml — This file has the data capture configurations.

  <?xml version="1.0" encoding="UTF-8" standalone="yes"?>
  <Javascript continueOnError="true" enabled="true" timeLimit="20" name="TraceableJS">
      <DisplayName>TraceableJS</DisplayName>
      <Properties>
          <Property name="ta_service_name">apigee</Property>
          <Property name="ta_data_capture_body_max_size_bytes">131072</Property>
          <Property name="ta_data_capture_http_body_request">true</Property>
          <Property name="ta_data_capture_http_body_response">true</Property>
          <Property name="ta_data_capture_http_headers_request">true</Property>
          <Property name="ta_data_capture_http_headers_response">true</Property>
          <Property name="ta_data_capture_target_request_response">true</Property>
          <Property name="ta_data_capture_allowed_types">json,graphql,x-www-form-urlencoded,xml</Property>
      </Properties>
      <ResourceURL>jsc://Traceable.js</ResourceURL>
  </Javascript>

Step 1 - Add Traceable agent for Apigee as Shared Flow

Complete the following steps to add Traceable's Shared Flow:

1. Login to your Apigee account.

2. Click on the Develop tab and navigate to SharedFlows.  

   

3. Add a new SharedFlow. Click on Upload Bundle. Navigate to the location where you have downloaded Traceable's Apigee policy. 

   

4. Upload the Traceable SharedFlow zip. You may wish to change the name of the file and import it.

5. Click on the imported SharedFlow and navigate to the Develop tab. 

   

6. Click on the TraceableSpansML and TraceableJS policy and update the configuration values. The sample files and configurations are listed above in the Policy bundle section. In TraceableSpansML, configuring the Traceable Platform agent IP address or hostname is mandatory. The other configurations are optional.

   

7. Click on Save.

8. Deploy the SharedFlow — Click on the Deploy button and select the environment to which you wish to deploy the agent. This deploys the agent as a SharedFlow to that environment. 

   

Traceable agent configurations

The following table lists the various Traceable agent configurations.

Step 2 - Attach the traceable agent

You can attach the Traceable agent to all the API proxies or a specific API proxy. Follow the steps in one of the following sections. Make a note of the following while attaching Traceable's Flow Callout policy. If you are attaching the policy directly to an API proxy, then make Traceable's Flow Callout policy as follows:

• Traceable policy should be the first one in Proxy → Request → Preflow

• Traceable policy should be the last one in Target → Request → Preflow

• Traceable policy should be the first one in Target → Response → Postflow

• Traceable policy should be the last one in Proxy → Response → Postflow

Option 1 - Attach Traceable agent to all API proxies using Flow Hooks

Complete the following steps to attach Traceable agent to all the API proxies:

1. Navigate to Admin Environment > Flow Hooks.

2. Edit the Flow Hooks and attach the traceable-agent sharedflow to Pre-proxy, Pre-target, Post-target, and Post-proxy Flow Hooks. This will attach Traceable agent with all API Proxies. 

   

You can now make requests to your API proxy. Traceable will capture these requests for analysis.

Option 2 - Attach Traceable agent to a specific API proxy

Complete the following steps to attach the Traceable agent to a specific API proxy:

1. Navigate to the API Proxy to which you wish to attach the traceable-apigee agent and click on the Develop tab.  

   

2. Click on Add a policy, the + sign beside Policies. 

   

3. Select Policy Type Flow Callout (refer to the above screenshot), update the names, and select traceable-shared-flow from the list of sharedflows.

4. In the Proxy Endpoints, click on PreFlow for the endpoint to which you want to attach the agent.

5. Click +Step for PreFlow on the request side in the flow diagram. 

   

6. In the Add Step dialogue box, click on the Existing Policy instance.

7. Select the Flow Callout Policy we added earlier and click on Add. 

   

8. Click on Postflow for the proxy endpoint.

9. Click +Step for Preflow on the response side in the flow diagram. 

   

10. In the Add Step dialogue box, click on the Existing Policy instance.

11. Select the Flow Callout Policy we added earlier and click on Add.

12. Repeat the above steps for Target endpoints while keeping the following in mind to attach the policy. 

    

    1. Traceable policy should be the last one in Target → Request → Postflow

    2. Traceable policy should be the first one in Target → Response → Preflow

13. Add a Fault Rule in both the Proxy Endpoints and Target Endpoints in XML as follows. Update the name of the flow callout policy.

    ActionScript ActionScript

    <FaultRules>
        <FaultRule name="traceable fault rule">
            <Step>
                <Name>NAME_OF_TRACEABLE_FLOW_CALLOUT_POLICY_ADDED_ABOVE</Name>
            </Step>
        </FaultRule>
    </FaultRules>

14. Click on Save the Configuration.

You can now make requests to your API proxy. Traceable will capture these requests for analysis.

(Optional) PostClientFlow instrumentation

1. Update the sharedflow default.xml as given below. Save and deploy the sharedflow.

2. Already attached to PreProxy, PreTarget, PostTarget, PostProxy. Now attach it to PostClientFlow as well.

Note

If you are attaching to each API, you should not have the sharedflow in flowhook. Therefore, remove it from all flowhooks if present.

<?xml version="1.0" encoding="UTF-8" standalone="yes"?>
<SharedFlow name="default"> 
    <Step>
        <Name>AssignPlatformValue</Name>
    </Step>
    <Step>
        <Condition>current.flow.name != "PostClientFlow"</Condition>
        <Name>TraceableJS</Name>
    </Step>
    <Step>
        <Condition>current.flow.name = "PostClientFlow"</Condition>
        <Name>ExportSpansML</Name>
    </Step>
</SharedFlow>

Reach out to the Traceable support team at support@traceable.ai for more help with this option.

Uninstall 

If you have attached a Traceable agent as a flow hook, remove it from the flow hook. If you have attached it to specific APIs, remove it from the APIs. Navigate to traceable sharedflow and click on Status. Click on Status > Undeploy. 

To delete it completely, navigate back to the SharedFlows Tab. In the list, delete traceable-sharedflow.

Troubleshooting

In Apigee UI, you can go to any API Proxy on which a Traceable agent is attached and click on the Traces/Debug tab. Start the trace session, send a request, and check the flow of requests and variables in TraceableJS policy on both the request and response side. Verify that the ExportSpansJS policy was executed on the Response side (not skipped). Make sure the connection from Apigee to the Traceable Platform is proper.