Find Traces

Traces are detailed breakdowns of the performance of requests in your application. They are collected at random via the configured sample rate and when a developer triggers them manually.

There are two ways to find interesting traces to identify bottlenecks or start improvements.

  1. Dive into one transaction in the list of transactions on the main application screen to immediately see a list of recent traces that where collected for this transaction.

  2. Select one of your manually triggered traces directly in the "My Callgraph Traces" box on the main application screen or follow the button "All Traces" for a list of all traces collected in the application.

Traces for a Transaction

If you monitor your application with Tideways in production, traces are collected for every individual transactions regularly and automatically. Start on the main screen to pick the transaction you want to investigate in detail:

Select a transaction to view a list of its traces

On the transaction details screen, you can then scroll to the bottom to see the "Profiling Traces" box with all the recently collected traces:

Pick a trace that was collected for this transaction

Select a trace and continue to Analyze Trace.

List of All Traces

The second way involves the "My Callgraph Traces" box on the main application screen. It shows a list of all the traces that you have triggered manually in the selected date range. You can directly pick one of the traces to jump to its details:

Pick a manually triggered trace for investigation

If you want to search and filter for trace details you can click on "All Traces" and find a list sorted by the most recently collected traces:

Filter and serach for trace details

You can filter the traces down by transaction, server and type of detail (Trace or Callgraph) and search for parts of the URL or transaction name. Sorting is currently not possible, but we are working on adding it again.

Advanced Query Language

The filters and selectors available in the Traces page are typically sufficient for normal use-cases. On top of that, there's now the possibility of filtering traces based on an easy-to-use, but powerful, query language. For you, that means:

  1. you can search for any specific values, and not only the pre-defined values that are provided in the selectors.
  2. you can combine different simpler expressions with logical operators to truly achieve your needs.

In the rest of this article, first to get an overview, some examples are provided. Then we delve into the available variables, value types, operators and how to combine them.


  • service = 'auth'
  • bottlenecks = 'http'
  • server in ["api", "replicate"]
  • transaction matches "controller"
  • response_time > 500
  • response_time < '1s'
  • release = "wheezy" and memory > 500
  • (response_time < 100 or response_time > 900) and = 'time' and sort.order = 'desc'
  • (date < '2017/03/01 13:00:00' and service = 'auth' and bottlenecks in ['gc', 'compile']) or (date >= '2017/03/01 13:00:00' and has_error = true)


Variables are the trace attributes that can be queried over. Naturally, only some operators and value types make sense for each variable.

Variable Operators Values
transaction =, in, matches string
response_time =, <, <=, >, >= int, string (int + ("ms" || "s"))
service =, in string
server =, in string
date =, <, <=, >, >= datetime
bottlenecks =, in "nplus1", "sql", "http", "cache", "gc", "session", "compile", "xdebug", "noisy"
memory =, <, <=, >, >= int
has_callgraph = boolean
user = string
has_error = boolean
release =, in string = "response_time", "time" or "memory"
sort.order = "desc" or "asc"
sql.count =, in, <, <=, >, >= int
sql.duration =, <, <=, >, >= int, string (int + ("ms" || "s"))
sql.max =, <, <=, >, >= int, string (int + ("ms" || "s"))
http.count =, in, <, <=, >, >= int
http.duration =, <, <=, >, >= int, string (int + ("ms" || "s"))
http.max =, <, <=, >, >= int, string (int + ("ms" || "s"))

Value Types

To specify the values you have in mind, several value types have been provided.

  • Int
  • Boolean: true or false
  • String: a string of characters, wrapped by ' or ", e.g. 'homepage' or "frontend"
  • Array: a list of ints or strings, enclosed by [], e.g. ["apiServer", "replicateServer"]

Please note that, a reference to scalar means any value type other than an array. Morever, date and datetime could be expressed with a string that adheres to the format Y-m-d H:i:s, e.g. '2017-02-15' or '2017-02-15 10:30:00'. Those variables which indicate a period of time (response_time, sql.duration, sql.max, http.duration, http.max) accept either an integer, which denotes the duration in milliseconds, or a string, which is a composition of the duration and the unit. Acceptable units are ms and s, e.g. 200, '200ms', '2s'.


Operators are the building blocks of the query language. Almost all of the operators are binary, accepting a variable in the before and a value after them. The type of the value depends on the operator.

Operator Value Type Description Example
= scalar Exact value equality check service = 'auth'
in array Value equals to one of the items in the array server in ["api", "replicate"]
matches string Value matches string transaction matches "controller"
<, <=, >, >= int Inequality comparisons response_time > 500
and operator Logical and release = "wheezy" and memory > 500
or operator Logical or response_time < 100 or response_time > 900

Still need help? Write [email protected] Write [email protected]