Traces
- 19 Dec 2023
- 9 Minutes to read
- PDF
Traces
- Updated on 19 Dec 2023
- 9 Minutes to read
- PDF
Article Summary
Share feedback
Thanks for sharing your feedback!
This topic explains Traces. You can do a system-wide search on Endpoint traces and spans using Traces. Traces provides various filter options for forensic analysis of traces and spans. Traces are a faster and more efficient way to carry out forensic analysis through traces and spans. In a normal scenario without Traces, you would have to search through every trace from the Traces tab on the API Endpoints details page or the Services page. A user sends a request and creates a trace in the system. The trace may traverse through multiple services to serve the request. A different endpoint may serve each service. In the journey of trace inside your infrastructure, it hops between many services served by different Endpoints. The part of the trace or a sub-segment of a trace that is related to a specific endpoint is called an Endpoint trace in the Traceable platform.
View traces from the Services or API Endpoint page
Click on one of the listed services on the Catalog → Services page. On the Services details page that is displayed, click on the Traces tab to view a list of Traces, as shown in the screenshot below:
.png "traceable_explorer_service_trace_list(1).png")
Traces view from the Services page
Click on an Endpoint, for example, GET/product/{product-id} to view the traces associated with it. The following screenshot shows the Endpoint trace for the above example.
Trace for an Endpoint
To see the full trace, click on the See Full Trace button. The following screenshot shows the full trace of the request.
You can view the traces similarly from the Traces tab of the API Endpoint details page.
View traces and spans
You can view the same trace from the Endpoint or Services page (as explained above) from the Traces page. The Traces page allows you to visualize the trace data based on various metrics. The Traceable platform receives raw span data from its agent. Traceable's business logic creates the trace and the Endpoint trace (the sub-segment of the trace) from these spans.
The Traces page also gives you the capability to view raw span data. Spans are the building blocks for a trace. Traceable captures all the request and response data; for example, request headers, body, cookies, and so on are captured as individual span attributes. These spans are put together by Traceable to build a trace. The Traces provides both the Endpoint Traces and Spans tab to search through.
The following screenshot shows all the Endpoint traces for the last 1-hour (selected from the top menu). You can filter or apply conditions to narrow down the search. Traceable provides many inbuilt filters. If a filter has an associated operation, it is displayed when you enter a space in the search bar.
.png "traceable_explorer_main_page(1).png")
You can also auto-fill the filter by clicking on the filter icon next to any value in the columns. For example, from the above screenshot, if you want to see all the Endpoint traces for the HTTP protocol, click on the filter icon next to HTTP.
.png "traceable_explorer_autofill_filter(2).png")
You can also combine multiple conditions; for example, the screenshot shows the output of an AND operation where protocol = HTTP AND Endpoint Name = GET /v1/data . All the filter conditions produce the result based only on the AND condition.
.png "traceable_explorer_autofill_filter_result(2).png")
You can carry out similar searches based on your investigative needs.
Saved queries
Traceable provides you with an option to save your queries for future use. Saved queries can help you save time if you use the same query again. For example, if you want to save the query in the above screenshot, click on the save icon as shown below:
.png "traceable_explorer_autofill_filter_result_save_query(1).png")
You can view your saved queries by clicking on the filter icon, as marked in the screenshot below.
.png "traceable_saved_queries_display(1).png")
Traceable provides a few inbuilt queries. The above screenshot shows a query showing the top APIs that see bot traffic with poor IP reputation from the Data Center. Similarly, there are other out-of-the-box queries. You can edit the default queries and save them with another name.
Saved queries with a time range
When you save a query, Traceable gives you three options for the time range you wish to save this query for. These are:
Inherit Application Time Range - This is equivalent to not choosing any time range.
Last 1 day - The time range displayed as the second option is the same that is chosen from the time-range drop-down at the top. In the example screenshot shown below, it is 1-day. It will be different if you choose a different time range. When you choose this option, the result of the query will be based on the saved time range. This means that if the application time range is 1 week and the saved query time range is 1 day, then the result would be based on the last 1 day's data.
Exact time - The last option saves the query result for the time when the query was run. For example, in the screenshot below, the query is run for 17/12/2023, 11:02:39 am - 18/12/2023, 11:02:39 am. When you save the query with this time range, it will always show the result between 17th and 18th December.
Note
The exact time range query result is not available for ever. If the chosen time range lies outside of the data retention period, the saved query does not display any result.
View the saved query detail
You can view the saved query details without running the query. Click on the filter icon and hover your pointer on the icon, as shown in the screenshot below.
Note the significance of these icons in the saved query:
.png "traceable_saved_query_timed_query_icon(1).png")
- This icon means that the query is saved with a time range.
- This icon means that the query was saved for a specific time range; however, that time range is now out of the data retention period.
.png "traceable_saved_query_timed_query_icon(1).png"){kind=small}
- This icon means that the query is saved with a time range.
- This icon means that the query was saved for a specific time range; however, that time range is now out of the data retention period.Visualization
You can also create custom visuals to compare against various metrics. One or more metrics can be added for the x and y-axis. The following screenshot gives an example of how you can use Trace visualization.
Traceable Visualization
In the above example, you are viewing the Average Duration metric that is grouped (in a group of 5) by Endpoint Name over a time interval of 1 minute. When you hover the mouse over the columns, a pop-up window shows the call duration for all five Endpoints. The visualization shows us that at 7:46 PM, the average call duration for POST/app/learnt/QKPmJNWoGI endpoint is 3ms. Similarly, you can create several visualizations to compare various parameters.
Operators
The following table lists the various operators with their descriptions and examples.
Operator | Description | Supported attribute types | Examples |
|---|---|---|---|
= | This operator tests for an exact match and is case-sensitive. When used against an array, it tests for at least one matching value. It can be used against a particular map entry by including a key value to test against to be included as part of the field declaration, but it cannot be used against a map itself. | String, Number, Boolean, Map Value, String Array |
|
!= | This operator is used as a negation of | String, Number, Boolean, Map Value, String Array |
|
IN | This operator is like | String, Number, Boolean, Map Value, String Array |
|
NOT_IN | Negation of | String, Number, Boolean, Map Value, String Array |
|
~ | Evaluates the provided regex for any matches. This operator is case-insensitive. | String, Map Value, String Array |
|
< | Less than | Number |
|
> | Greater than | Number |
|
<= | Less than or equal to | Number |
|
>= | Greater than or equal to | Number |
|
CONTAINS | This operator checks if the string contains the provided substring. The operator is case-insensitive. For arrays, the operator checks if there are any matching values. | String, String Array |
|
CONTAINS_KEY | This operator checks if the map contains a key with the provided name. The operator is case-sensitive. | Map |
|
CONTAINS_KEY_LIKE | This operator checks if the map contains a key matching the provided regex. The operator is case-insensitive. | Map |
|
NOT_CONTAINS_KEY | This operator checks if the map contains no key that exactly matches the provided value. The operator is case-sensitive. | Map |
|
Filter attributes
The following tables explain a few of the filter attributes for Endpoint traces and spans.
Endpoint Traces
Attribute | Definition |
|---|---|
API exit call | Number of outbound calls made by the API Endpoint |
Calls | Number of calls to the API Endpoint |
Duration | Time spent within the boundary of a service containing the API Endpoint is the time between the entry span into the service and the exit span out from the service. |
End time | The end time of the segment of the trace |
Endpoint ID | The ID of the API Endpoint this trace segment belongs to |
Endpoint Name | Name of the API Endpoint this trace segment belongs to |
Endpoint Trace ID | The span ID corresponds to the entry span, which is the starting span of the Endpoint trace segment. |
Environment | All the data visible is scoped under the selected environment. But if you select AllEnvironments, this shows the environment corresponding to the trace segment. |
Error count | Count of errors in the given API trace |
HTTP Method | HTTP Methods, for example, GET/POST/PUT/DELETE, and so on. |
IP Address | Client's IP address |
Is Endpoint PII | True if the endpoint corresponding to the trace segment has any sensitive information/parameters. |
Protocol | HTTP or GRPC |
Request Body | Contains the API request for the given API |
Request Content-Type | Content-type request header |
Service ID | The ID of the Service that owns the API Endpoint trace this segment belongs to |
Service Name | Name of the Service that owns the API Endpoint this trace segment belongs to |
Session ID | User session ID in which the current operation (that resulted in the trace) was performed |
Start time | The start time of the segment of the trace |
Trace ID | The trace ID of the trace that this segment is part of |
User ID | The username or client_id was extracted from the access token or Authorization header in the inbound request from outside the system. |
User Role | The client's actual or assumed role is making the request extracted from the token or application security context. |
User Scope | Scopes or granted authorities the client currently possesses were extracted from the token or application security context. The value would come from the scope associated with an OAuth 2.0 Access Token or an attribute value in a SAML 2.0 Assertion. |
SPANS
Attribute | Definition |
|---|---|
Duration | The time duration for the span |
Endtime | The finish time for the span |
Error Count | Count the number of error occurrences in the given span |
Parent Span ID | Span ID for the parent span |
Protocol | One of HTTP, HTTPS, GRPC, REDIS, MONGO, JDBC, RABBIT_MQ protocol |
Request Body | Contains the API request for the given API |
Request Content-Type | Content-type request header |
Request Params | The request parameters |
Request size | The size of the request |
Request Headers | The request headers |
Response Content Type | Content-type response header |
Response Headers | The response headers |
Response Size | The size of the response |
Service ID | The ID of the service corresponding to the given span |
Service Name | Name of the service corresponding to the given span |
Span ID | The identifier for the span |
Span Name | The name for the span |
Start Time | The time when the span event started |
Tags | The key-value attributes for the span. It captures various events or metadata about the span |
Trace ID | An Identifier for the trace that this span belongs to |
Was this article helpful?
