435 lines
11 KiB
Markdown
435 lines
11 KiB
Markdown
# Quickstart: SQL-based analysis and trace-based metrics
|
|
|
|
_This quickstart explains how to use `trace_processor` as well as its Python API to
|
|
programmatically query the trace contents through SQL and compute trace-based metrics._
|
|
|
|
## Trace Processor
|
|
|
|
TraceProcessor is a multi-format trace importing and query engine based on
|
|
SQLite. It comes both as a C++ library and as a standalone executable:
|
|
`trace_processor_shell` (or just `trace_processor`).
|
|
|
|
### Setup
|
|
|
|
```bash
|
|
# Download prebuilts (Linux and Mac only)
|
|
curl -LO https://get.perfetto.dev/trace_processor
|
|
chmod +x ./trace_processor
|
|
|
|
# Start the interactive shell
|
|
./trace_processor trace.perfetto-trace
|
|
|
|
# Start a local trace processor instance to replace wasm module in the UI
|
|
./trace_processor trace.perfetto-trace --http
|
|
```
|
|
|
|
NOTE: In HTTP mode the trace will be loaded into the `trace_processor` and
|
|
the UI will connect and issue queries over TCP. This can allow
|
|
arbitrary sized traces to be loaded since there are no memory
|
|
constraints, unlike the WASM module. In addition, this can improve
|
|
performance in the UI as it issues SQL queries.
|
|
|
|
See [Trace Processor docs](/docs/analysis/trace-processor.md) for the full
|
|
TraceProcessor guide.
|
|
|
|
### Sample queries
|
|
|
|
For more exhaustive examples see the _SQL_ section of the various _Data sources_
|
|
docs.
|
|
|
|
#### Slices
|
|
|
|
Slices are stackable events which have name and span some duration of time.
|
|
|
|
|
|
|
|
```
|
|
> SELECT ts, dur, name FROM slice
|
|
ts dur name
|
|
-------------------- -------------------- ---------------------------
|
|
261187017446933 358594 eglSwapBuffersWithDamageKHR
|
|
261187017518340 357 onMessageReceived
|
|
261187020825163 9948 queueBuffer
|
|
261187021345235 642 bufferLoad
|
|
261187121345235 153 query
|
|
...
|
|
```
|
|
|
|
#### Counters
|
|
|
|
Counters are events with a value which changes over time.
|
|
|
|
|
|
|
|
```
|
|
> SELECT ts, value FROM counter
|
|
ts value
|
|
-------------------- --------------------
|
|
261187012149954 1454.000000
|
|
261187012399172 4232.000000
|
|
261187012447402 14304.000000
|
|
261187012535839 15490.000000
|
|
261187012590890 17490.000000
|
|
261187012590890 16590.000000
|
|
...
|
|
```
|
|
|
|
#### Scheduler slices
|
|
|
|
Scheduler slices indicate which thread was scheduled on which CPU at which time.
|
|
|
|
|
|
|
|
```
|
|
> SELECT ts, dur, cpu, utid FROM sched
|
|
ts dur cpu utid
|
|
-------------------- -------------------- -------------------- --------------------
|
|
261187012170489 267188 0 390
|
|
261187012170995 247153 1 767
|
|
261187012418183 12812 2 2790
|
|
261187012421099 220000 6 683
|
|
261187012430995 72396 7 2791
|
|
...
|
|
```
|
|
|
|
### Trace-based metrics
|
|
|
|
Trace Processor offers also a higher-level query interface that allows to run
|
|
pre-baked queries, herein called "metrics". Metrics are generally curated by
|
|
domain experts, often the same people who add the instrumentation points in the
|
|
first place, and output structured JSON/Protobuf/text.
|
|
Metrics allow to get a summarized view of the trace without having to type any
|
|
SQL or having to load the trace in the UI.
|
|
|
|
The metrics` schema files live in the
|
|
[/protos/perfetto/metrics](/protos/perfetto/metrics/) directory.
|
|
The corresponding SQL queries live in
|
|
[/src/trace_processor/metrics](/src/trace_processor/metrics/).
|
|
|
|
#### Run a single metric
|
|
|
|
Let's run the [`android_cpu`](/protos/perfetto/metrics/android/cpu_metric.proto)
|
|
metric. This metrics computes the total CPU time and the total cycles
|
|
(CPU frequency * time spent running at that frequency) for each process in the
|
|
trace, breaking it down by CPU (_core_) number.
|
|
|
|
```protobuf
|
|
./trace_processor --run-metrics android_cpu trace.perfetto-trace
|
|
|
|
android_cpu {
|
|
process_info {
|
|
name: "/system/bin/init"
|
|
threads {
|
|
name: "init"
|
|
core {
|
|
id: 1
|
|
metrics {
|
|
mcycles: 1
|
|
runtime_ns: 570365
|
|
min_freq_khz: 1900800
|
|
max_freq_khz: 1900800
|
|
avg_freq_khz: 1902017
|
|
}
|
|
}
|
|
core {
|
|
id: 3
|
|
metrics {
|
|
mcycles: 0
|
|
runtime_ns: 366406
|
|
min_freq_khz: 1900800
|
|
max_freq_khz: 1900800
|
|
avg_freq_khz: 1902908
|
|
}
|
|
}
|
|
...
|
|
}
|
|
...
|
|
}
|
|
process_info {
|
|
name: "/system/bin/logd"
|
|
threads {
|
|
name: "logd.writer"
|
|
core {
|
|
id: 0
|
|
metrics {
|
|
mcycles: 8
|
|
runtime_ns: 33842357
|
|
min_freq_khz: 595200
|
|
max_freq_khz: 1900800
|
|
avg_freq_khz: 1891825
|
|
}
|
|
}
|
|
core {
|
|
id: 1
|
|
metrics {
|
|
mcycles: 9
|
|
runtime_ns: 36019300
|
|
min_freq_khz: 1171200
|
|
max_freq_khz: 1900800
|
|
avg_freq_khz: 1887969
|
|
}
|
|
}
|
|
...
|
|
}
|
|
...
|
|
}
|
|
...
|
|
}
|
|
```
|
|
|
|
#### Running multiple metrics
|
|
|
|
Multiple metrics can be flagged using comma separators to the `--run-metrics`
|
|
flag. This will output a text proto with the combined result of running both
|
|
metrics.
|
|
|
|
```protobuf
|
|
$ ./trace_processor --run-metrics android_mem,android_cpu trace.perfetto-trace
|
|
|
|
android_mem {
|
|
process_metrics {
|
|
process_name: ".dataservices"
|
|
total_counters {
|
|
anon_rss {
|
|
min: 19451904
|
|
max: 19890176
|
|
avg: 19837548.157829277
|
|
}
|
|
file_rss {
|
|
min: 25804800
|
|
max: 25829376
|
|
avg: 25827909.957489081
|
|
}
|
|
swap {
|
|
min: 9289728
|
|
max: 9728000
|
|
avg: 9342355.8421707246
|
|
}
|
|
anon_and_swap {
|
|
min: 29179904
|
|
max: 29179904
|
|
avg: 29179904
|
|
}
|
|
}
|
|
...
|
|
}
|
|
...
|
|
}
|
|
android_cpu {
|
|
process_info {
|
|
name: "/system/bin/init"
|
|
threads {
|
|
name: "init"
|
|
core {
|
|
id: 1
|
|
metrics {
|
|
mcycles: 1
|
|
runtime_ns: 570365
|
|
min_freq_khz: 1900800
|
|
max_freq_khz: 1900800
|
|
avg_freq_khz: 1902017
|
|
}
|
|
}
|
|
...
|
|
}
|
|
...
|
|
}
|
|
...
|
|
}
|
|
```
|
|
|
|
#### JSON and binary output
|
|
|
|
The trace processor also supports binary protobuf and JSON as alternative output
|
|
formats. This is useful when the intended reader is an offline tool.
|
|
|
|
Both single and multiple metrics are supported as with proto text output.
|
|
|
|
```
|
|
./trace_processor --run-metrics android_mem --metrics-output=binary trace.perfetto-trace
|
|
<binary protobuf output>
|
|
|
|
./trace_processor --run-metrics android_mem,android_cpu --metrics-output=json trace.perfetto-trace
|
|
{
|
|
"android_mem": {
|
|
"process_metrics": [
|
|
{
|
|
"process_name": ".dataservices",
|
|
"total_counters": {
|
|
"anon_rss": {
|
|
"min": 19451904.000000,
|
|
"max": 19890176.000000,
|
|
"avg": 19837548.157829
|
|
},
|
|
"file_rss": {
|
|
"min": 25804800.000000,
|
|
"max": 25829376.000000,
|
|
"avg": 25827909.957489
|
|
},
|
|
"swap": {
|
|
"min": 9289728.000000,
|
|
"max": 9728000.000000,
|
|
"avg": 9342355.842171
|
|
},
|
|
"anon_and_swap": {
|
|
"min": 29179904.000000,
|
|
"max": 29179904.000000,
|
|
"avg": 29179904.000000
|
|
}
|
|
},
|
|
...
|
|
},
|
|
...
|
|
]
|
|
}
|
|
"android_cpu": {
|
|
"process_info": [
|
|
{
|
|
"name": "\/system\/bin\/init",
|
|
"threads": [
|
|
{
|
|
"name": "init",
|
|
"core": [
|
|
{
|
|
"id": 1,
|
|
"metrics": {
|
|
"mcycles": 1,
|
|
"runtime_ns": 570365,
|
|
"min_freq_khz": 1900800,
|
|
"max_freq_khz": 1900800,
|
|
"avg_freq_khz": 1902017
|
|
}
|
|
},
|
|
...
|
|
]
|
|
...
|
|
}
|
|
...
|
|
]
|
|
...
|
|
},
|
|
...
|
|
]
|
|
...
|
|
}
|
|
}
|
|
```
|
|
|
|
## Python API
|
|
|
|
The API can be run without requiring the `trace_processor` binary to be
|
|
downloaded or installed.
|
|
|
|
### Setup
|
|
```
|
|
$ pip install perfetto
|
|
```
|
|
NOTE: The API is only compatible with Python3.
|
|
|
|
### Example functions
|
|
See the Python API section of
|
|
[Trace Processor (SQL)](/docs/analysis/trace-processor.md) to get
|
|
more details on all available functions.
|
|
|
|
#### Query
|
|
```python
|
|
from perfetto.trace_processor import TraceProcessor
|
|
tp = TraceProcessor(trace='trace.perfetto-trace')
|
|
|
|
qr_it = tp.query('SELECT name FROM slice')
|
|
for row in qr_it:
|
|
print(row.name)
|
|
```
|
|
**Output**
|
|
```
|
|
eglSwapBuffersWithDamageKHR
|
|
onMessageReceived
|
|
queueBuffer
|
|
bufferLoad
|
|
query
|
|
...
|
|
```
|
|
#### Query as Pandas DataFrame
|
|
```python
|
|
from perfetto.trace_processor import TraceProcessor
|
|
tp = TraceProcessor(trace='trace.perfetto-trace')
|
|
|
|
qr_it = tp.query('SELECT ts, name FROM slice')
|
|
qr_df = qr_it.as_pandas_dataframe()
|
|
print(qr_df.to_string())
|
|
```
|
|
**Output**
|
|
```
|
|
ts name
|
|
-------------------- ---------------------------
|
|
261187017446933 eglSwapBuffersWithDamageKHR
|
|
261187017518340 onMessageReceived
|
|
261187020825163 queueBuffer
|
|
261187021345235 bufferLoad
|
|
261187121345235 query
|
|
...
|
|
```
|
|
#### Metric
|
|
```python
|
|
from perfetto.trace_processor import TraceProcessor
|
|
tp = TraceProcessor(trace='trace.perfetto-trace')
|
|
|
|
cpu_metrics = tp.metric(['android_cpu'])
|
|
print(cpu_metrics)
|
|
```
|
|
**Output**
|
|
```
|
|
metrics {
|
|
android_cpu {
|
|
process_info {
|
|
name: "/system/bin/init"
|
|
threads {
|
|
name: "init"
|
|
core {
|
|
id: 1
|
|
metrics {
|
|
mcycles: 1
|
|
runtime_ns: 570365
|
|
min_freq_khz: 1900800
|
|
max_freq_khz: 1900800
|
|
avg_freq_khz: 1902017
|
|
}
|
|
}
|
|
core {
|
|
id: 3
|
|
metrics {
|
|
mcycles: 0
|
|
runtime_ns: 366406
|
|
min_freq_khz: 1900800
|
|
max_freq_khz: 1900800
|
|
avg_freq_khz: 1902908
|
|
}
|
|
}
|
|
...
|
|
}
|
|
...
|
|
}
|
|
...
|
|
}
|
|
}
|
|
```
|
|
|
|
## Next steps
|
|
|
|
There are several options for exploring more of the trace analysis features
|
|
Perfetto provides:
|
|
|
|
* The [trace conversion quickstart](/docs/quickstart/traceconv.md) gives an
|
|
overview on how to convert Perfetto traces to legacy formats to integrate with
|
|
existing tooling.
|
|
* The [Trace Processor documentation](/docs/analysis/trace-processor.md) gives
|
|
more information about how to work with trace processor including details on
|
|
how to write queries and how tables in trace processor are organized.
|
|
* The [metrics documentation](/docs/analysis/metrics.md) gives a more in-depth
|
|
look into metrics including a short walkthrough on how to build an
|
|
experimental metric from scratch.
|
|
* The [SQL table reference](/docs/analysis/sql-tables.autogen) gives a
|
|
comprehensive guide to the all the available tables in trace processor.
|
|
* The [common tasks](/docs/contributing/common-tasks.md) page gives a list of
|
|
steps on how new metrics can be added to the trace processor.
|