167 lines
5.3 KiB
Markdown
167 lines
5.3 KiB
Markdown
# Profcollect
|
|
|
|
Profcollect is a system daemon that facilitates sampling profile collection and reporting for native
|
|
platform applications.
|
|
|
|
Profcollect can only be enabled on `userdebug` or `eng` builds.
|
|
|
|
## Supported Platforms
|
|
|
|
Currently Profcollect only supports collecting profiles from Coresight ETM enabled ARM devices.
|
|
|
|
Instructions to enable Coresight ETM can be found from the
|
|
[simpleperf manual](https://android.googlesource.com/platform/system/extras/+/refs/heads/master/simpleperf/doc/collect_etm_data_for_autofdo.md).
|
|
|
|
## Usage
|
|
|
|
Profcollect has two components: `profcollectd`, the system daemon, and `profcollectctl`, the command
|
|
line interface.
|
|
|
|
### Collection
|
|
|
|
`profcollectd` can be started from `adb` directly (under root), or automatically on system boot by
|
|
setting system property through:
|
|
|
|
```
|
|
adb shell device_config put profcollect_native_boot enabled true
|
|
```
|
|
|
|
Profcollect collects profiles periodically, as well as through triggers like app launch events. Only
|
|
a percentage of these events result in a profile collection to avoid using too much resource, these
|
|
are controlled by the following configurations:
|
|
|
|
| Event | Config |
|
|
|------------|------------------------|
|
|
| Periodic | collection\_interval |
|
|
| App launch | applaunch\_trace\_freq |
|
|
|
|
Setting the frequency value to `0` disables collection for the corresponding event.
|
|
|
|
#### Custom configuration
|
|
|
|
In adb root:
|
|
|
|
```
|
|
# Record every 60s (By default, record every 10m). The actual interval will be longer than the
|
|
# set value if the device goes to hibernation.
|
|
oriole:/ # setprop persist.device_config.profcollect_native_boot.collection_interval 60
|
|
|
|
# Each time recording, record ETM data for 1s (By default, it's 0.5s).
|
|
oriole:/ # setprop persist.device_config.profcollect_native_boot.sampling_period 1000
|
|
|
|
# Set ETM data storage limit to 50G (By default, it is 512M).
|
|
oriole:/ # setprop persist.device_config.profcollect_native_boot.max_trace_limit 53687091200
|
|
|
|
# Enable ETM data collection (By default, it's decided by the server).
|
|
oriole:/ # setprop persist.device_config.profcollect_native_boot.enabled true
|
|
|
|
# After adjusting configuration, need to restart profcollectd
|
|
oriole:/ # setprop ctl.stop profcollectd
|
|
# Wait for a few seconds.
|
|
oriole:/ # setprop ctl.start profcollectd
|
|
|
|
# Check if profcollectd is running
|
|
oriole:/ # ps -e | grep profcollectd
|
|
root 918 1 10945660 47040 binder_wait_for_work 0 S profcollectd
|
|
|
|
# Check if the new configuration takes effect.
|
|
oriole:/ # cat /data/misc/profcollectd/output/config.json
|
|
{"version":1,"node_id":[189,15,145,225,97,167],"build_fingerprint":"google/oriole/oriole:Tiramisu/TP1A.220223.002/8211650:userdebug/dev-keys","collection_interval":{"secs":60,"nanos":0},"sampling_period":{"secs":1,"nanos":0},"binary_filter":"^/(system|apex/.+)/(bin|lib|lib64)/.+","max_trace_limit":53687091200}
|
|
```
|
|
|
|
To check existing collected ETM data:
|
|
```
|
|
oriole:/ # cd data/misc/profcollectd/trace/
|
|
oriole:/data/misc/profcollectd/trace # ls
|
|
```
|
|
|
|
To check if ETM data can be collected successfully:
|
|
```
|
|
# Trigger one collection manually.
|
|
oriole:/ # profcollectctl once
|
|
Trace once
|
|
|
|
# Check trace directory to see if there is a recent manual trace file.
|
|
oriole:/ # ls /data/misc/profcollectd/trace/
|
|
20220224-222946_manual.etmtrace
|
|
```
|
|
|
|
If there are too many trace files, we need to processing them to avoid reaching storage limit.
|
|
It may take a long time.
|
|
```
|
|
oriole:/ # profcollectctl process
|
|
Processing traces
|
|
```
|
|
|
|
### Processing
|
|
|
|
The raw tracing data needs to be combined with the original binary to create the AutoFDO branch
|
|
list. This is a costly process, thus it is done separately from the profile collection. Profcollect
|
|
attempts to process all the traces when the device is idle and connected to a power supply. It can
|
|
also be initiated by running:
|
|
|
|
```
|
|
adb shell profcollectctl process
|
|
```
|
|
|
|
### Reporting
|
|
|
|
#### Manual
|
|
|
|
After actively using the device for a period of time, the device should have gathered enough data to
|
|
generate a good quality PGO profile that represents typical system usage. Run the following command
|
|
to create a profile report:
|
|
|
|
```
|
|
$ adb shell profcollectctl report
|
|
Creating profile report
|
|
Report created at: 12345678-0000-abcd-8000-12345678abcd
|
|
```
|
|
|
|
You can then fetch the report by running (under root):
|
|
|
|
```
|
|
adb pull /data/misc/profcollectd/report/12345678-0000-abcd-8000-12345678abcd.zip
|
|
```
|
|
|
|
#### Automated Uploading to Server
|
|
|
|
*In development*
|
|
|
|
### Post Processing
|
|
|
|
For each trace file, run:
|
|
|
|
```
|
|
simpleperf inject \
|
|
-i {TRACE_FILE_NAME} \
|
|
-o {OUTPUT_FILE_NAME}.data \
|
|
--binary {BINARY_NAME} \
|
|
--symdir out/target/product/{PRODUCT_NAME}/symbols
|
|
```
|
|
|
|
Afterwards, run [AutoFDO](https://github.com/google/autofdo) to generate Clang PGO profiles:
|
|
|
|
```
|
|
create_llvm_prof \
|
|
--profiler text \
|
|
--binary=${BINARY_PATH} \
|
|
--profile=${INPUT_FILE_NAME} \
|
|
--out={OUTPUT_FILE_NAME}.profdata
|
|
```
|
|
|
|
Finally, merge all the PGO profiles into one profile:
|
|
|
|
```
|
|
find {INPUT_DIR} -name *.profdata > proflist
|
|
prebuilts/clang/host/linux-x86/llvm-binutils-stable/llvm-profdata merge \
|
|
--binary \
|
|
--sample \
|
|
--input-files proflist \
|
|
--output merged.profdata
|
|
```
|
|
|
|
More profile data usually generates better quality profiles. You may combine data from multiple
|
|
devices running the same build to improve profile quality, and/or reduce the performance impact for
|
|
each device (by reducing collection frequency).
|