1077 lines
39 KiB
ReStructuredText
1077 lines
39 KiB
ReStructuredText
.. _docs-build-system:
|
|
|
|
============
|
|
Build system
|
|
============
|
|
Building software for embedded devices is a complex process. Projects often have
|
|
custom toolchains, target different hardware platforms, and require additional
|
|
configuration and post-processing of artifacts.
|
|
|
|
As a modern embedded framework, Pigweed's goal is to collect these embedded use
|
|
cases into a powerful and flexible build system, then extend it with support for
|
|
modern software development practices.
|
|
|
|
See :ref:`docs-python-build` for information about Python build automation with
|
|
Pigweed.
|
|
|
|
.. toctree::
|
|
:hidden:
|
|
|
|
python_build
|
|
|
|
What's in a build system?
|
|
=========================
|
|
A quality build system provides a variety of features beyond compiling code.
|
|
Throughout our experience with embedded development, we've found several build
|
|
features to be especially useful, and designed Pigweed's build system with them
|
|
in mind.
|
|
|
|
Simple toolchain configuration
|
|
------------------------------
|
|
Embedded projects often use custom build toolchains for their specific hardware.
|
|
Configuring these should be a simple process, both in their initial setup and
|
|
later adjustments.
|
|
|
|
Multi-target builds
|
|
-------------------
|
|
Virtually every consumer product has firmware that targets different boards or
|
|
MCUs during development. While building for a single board is simple enough, the
|
|
complexity of supporting different targets ranges from changing compiler flags
|
|
to swapping out entire libraries of firmware and drivers. This is often done by
|
|
running multiple builds, configuring each one accordingly. In Pigweed, we've
|
|
designed our build system with first-class multi-target support in mind,
|
|
allowing any number of target configurations to be built simultaneously.
|
|
|
|
Multi-language support
|
|
----------------------
|
|
Embedded projects are typically written in C, C++, and assembly. However, it is
|
|
possible to have firmware written in other languages, such as Rust.
|
|
Additionally, projects may have host-side tooling written in a wide variety of
|
|
languages. Having all of these build together proves to be a large time saver.
|
|
|
|
Custom scripting
|
|
----------------
|
|
Embedded projects often require post-processing of build artifacts; these may
|
|
include:
|
|
|
|
* Extracting ELF sections into a different container
|
|
* Injecting metadata into firmware images
|
|
* Image signing
|
|
* Creating databases of symbols for debugging
|
|
* Extracting string tokens into a database (for example, with
|
|
:ref:`module-pw_tokenizer`)
|
|
|
|
These are run as steps during a build, facilitated by the build system.
|
|
|
|
See also
|
|
^^^^^^^^
|
|
|
|
* :ref:`module-pw_build-python-action`
|
|
|
|
Python
|
|
------
|
|
Python is a favorite scripting language of many development teams, and here at
|
|
Pigweed, we're no exception. Much of Pigweed's host-side tooling is written in
|
|
Python. While Python works great for local development, problems can arise when
|
|
scripts need to be packaged and distributed for vendors or factory teams. Having
|
|
proper support for packaging Python within a build system allows teams to focus
|
|
on writing code instead of worrying about distribution.
|
|
|
|
Size reporting
|
|
--------------
|
|
On embedded devices, memory is everything. Most projects have some sort of
|
|
custom tooling to determine how much flash and RAM space their firmware uses.
|
|
Being able to run size reports as part of a build ensures that they are always
|
|
up-to-date and allows space usage to be tracked over time.
|
|
|
|
See also
|
|
^^^^^^^^
|
|
|
|
* :ref:`module-pw_bloat`
|
|
|
|
Documentation
|
|
-------------
|
|
An oft-neglected part of software development, documentation is invaluable for
|
|
future maintainers of a project. As such, Pigweed has integrated documentation
|
|
which builds alongside its code and combines with other build features, such as
|
|
size reports, to provide high quality, up-to-date references for developers.
|
|
|
|
See also
|
|
^^^^^^^^
|
|
|
|
* :ref:`module-pw_docgen`
|
|
|
|
Unit testing
|
|
------------
|
|
Unit tests are essential to ensure that the functionality of code remains
|
|
consistent as changes are made to avoid accidental regressions. Running unit
|
|
tests as part of a build keeps developers constantly aware of the impact of
|
|
their changes.
|
|
|
|
Host-side unit tests
|
|
^^^^^^^^^^^^^^^^^^^^
|
|
Though Pigweed targets embedded devices, a lot of its code can be run and tested
|
|
on a host desktop by swapping out backends to host platform libraries. This is
|
|
highly beneficial during development, as it allows tests to consistently run
|
|
without having to go through the process of flashing a device.
|
|
|
|
Device-side unit tests
|
|
^^^^^^^^^^^^^^^^^^^^^^
|
|
As useful as host-side tests are, they are not sufficient for developing actual
|
|
firmware, and it is critical to run tests on the actual hardware. Pigweed has
|
|
invested into creating a test framework and build integration for running tests
|
|
across physical devices as part of a build.
|
|
|
|
See also
|
|
^^^^^^^^
|
|
|
|
* :ref:`module-pw_unit_test`
|
|
* :ref:`module-pw_target_runner`
|
|
|
|
Bonus: pw watch
|
|
---------------
|
|
In web development, it is common to have a file system watcher listening for
|
|
source file changes and triggering a build for quick iteration. When combined
|
|
with a fast incremental build system, this becomes a powerful feature, allowing
|
|
things such as unit tests and size reports to re-run whenever any dependent
|
|
code is modified.
|
|
|
|
While initially seen as somewhat of a gimmick, Pigweed's watcher has become a
|
|
staple of Pigweed development, with most Pigweed users having it permanently
|
|
running in a terminal window.
|
|
|
|
See also
|
|
^^^^^^^^
|
|
|
|
* :ref:`module-pw_watch`
|
|
|
|
Pigweed's build systems
|
|
=======================
|
|
Pigweed can be used either as a monolith or à la carte, slotting into an
|
|
existing project. To this end, Pigweed supports multiple build systems, allowing
|
|
Pigweed-based projects to choose the most suitable one for them.
|
|
|
|
Of the supported build systems, GN is the most full-featured, followed by CMake,
|
|
and finally Bazel.
|
|
|
|
.. note::
|
|
A quick note on terminology: the word "target" is overloaded within GN/Bazel (and
|
|
Pigweed)---it can refer to either a GN/Bazel build target, such as a ``source_set``
|
|
or ``executable``, or to an output platform (e.g. a specific board, device, or
|
|
system).
|
|
|
|
To avoid confusing the two, we refer to the former as "GN/Bazel targets" and the
|
|
latter as "Pigweed targets".
|
|
|
|
GN
|
|
--
|
|
A perhaps unfamiliar name, `GN (Generate Ninja)`_ is a meta-build system that
|
|
outputs `Ninja`_ build files, originally designed for use in Chromium. Pigweed
|
|
first experimented with GN after hearing about it from another team, and we
|
|
quickly came to appreciate its speed and simplicity. GN has become Pigweed's
|
|
primary build system; it is used for all upstream development and strongly
|
|
recommended for Pigweed-based projects where possible.
|
|
|
|
.. _CMake: https://cmake.org/
|
|
.. _Bazel: https://bazel.build/
|
|
.. _GN (Generate Ninja): https://gn.googlesource.com/gn
|
|
.. _Ninja: https://ninja-build.org/
|
|
|
|
The GN build
|
|
============
|
|
This section describes Pigweed's GN build structure, how it is used upstream,
|
|
build conventions, and recommendations for Pigweed-based projects. While
|
|
containing some details about how GN works in general, this section is not
|
|
intended to be a guide on how to use GN. To learn more about the tool itself,
|
|
refer to the official `GN reference`_.
|
|
|
|
.. _GN reference: https://gn.googlesource.com/gn/+/HEAD/docs/reference.md
|
|
|
|
Entrypoint: .gn
|
|
---------------
|
|
The entrypoint to a GN build is the ``.gn`` file, which defines a project's root
|
|
directory (henceforth ``//``).
|
|
|
|
``.gn`` must point to the location of a ``BUILDCONFIG.gn`` file for the project.
|
|
In Pigweed upstream, this is its only purpose.
|
|
|
|
Downstream projects may additionally use ``.gn`` to set global overrides for
|
|
Pigweed's build arguments, which apply across all Pigweed targets. For example,
|
|
a project could configure the protobuf libraries that it uses. This is done by
|
|
defining a ``default_args`` scope containing the overrides.
|
|
|
|
.. code::
|
|
|
|
# The location of the BUILDCONFIG file.
|
|
buildconfig = "//BUILDCONFIG.gn"
|
|
|
|
# Build arguments set across all Pigweed targets.
|
|
default_args = {
|
|
dir_pw_third_party_nanopb = "//third_party/nanopb-0.4.2"
|
|
}
|
|
|
|
Configuration: BUILDCONFIG.gn
|
|
-----------------------------
|
|
The file ``BUILDCONFIG.gn`` configures the GN build by defining any desired
|
|
global variables/options. It can be located anywhere in the build tree, but is
|
|
conventionally placed at the root. ``.gn`` points GN to this file.
|
|
|
|
``BUILDCONFIG.gn`` is evaluated before any other GN files, and variables defined
|
|
within it are placed into GN's global scope, becoming available in every file
|
|
without requiring imports.
|
|
|
|
The options configured in this file differ from those in ``.gn`` in two ways:
|
|
|
|
1. ``BUILDCONFIG.gn`` is evaluated for every GN toolchain (and Pigweed target),
|
|
whereas ``.gn`` is only evaluated once. This allows ``BUILDCONFIG.gn`` to set
|
|
different options for each Pigweed target.
|
|
2. In ``.gn``, only GN build arguments can be overridden. ``BUILDCONFIG.gn``
|
|
allows defining arbitrary variables.
|
|
|
|
Generally, it is preferable to expose configuration options through build args
|
|
instead of globals in ``BUILDCONFIG.gn`` (something Pigweed's build previously
|
|
did), as they are more flexible, greppable, and easier to manage. However, it
|
|
may make sense to define project-specific constants in ``BUILDCONFIG.gn``.
|
|
|
|
Pigweed's upstream ``BUILDCONFIG.gn`` does not define any variables; it just
|
|
sets Pigweed's default toolchain, which GN requires.
|
|
|
|
.. _top-level-build:
|
|
|
|
Top-level GN targets: //BUILD.gn
|
|
--------------------------------
|
|
The root ``BUILD.gn`` file defines all of the libraries, images, tests, and
|
|
binaries built by a Pigweed project. This file is evaluated immediately after
|
|
``BUILDCONFIG.gn``, with the active toolchain (which is the default toolchain
|
|
at the start of a build).
|
|
|
|
``//BUILD.gn`` is responsible for enumerating each of the Pigweed targets built
|
|
by a project. This is done by instantiating a version of each of the project's
|
|
GN target groups with each Pigweed target's toolchain. For example, in upstream,
|
|
all of Pigweed's GN targets are contained within the ``pigweed_default`` group.
|
|
This group is instantiated multiple times, with different Pigweed target
|
|
toolchains.
|
|
|
|
These groups include the following:
|
|
|
|
* ``host`` -- builds ``pigweed_default`` with Clang or GCC, depending on the
|
|
platform
|
|
* ``host_clang`` -- builds ``pigweed_default`` for the host with Clang
|
|
* ``host_gcc`` -- builds ``pigweed_default`` for the host with GCC
|
|
* ``stm32f429i`` -- builds ``pigweed_default`` for STM32F429i Discovery board
|
|
* ``docs`` -- builds the Pigweed documentation and size reports
|
|
|
|
Pigweed projects are recommended to follow this pattern, creating a top-level
|
|
group for each of their Pigweed targets that builds a common GN target with the
|
|
appropriate toolchain.
|
|
|
|
It is important that no dependencies are listed under the default toolchain
|
|
within ``//BUILD.gn``, as it does not configure any build parameters, and
|
|
therefore should not evaluate any other GN files. The pattern that Pigweed uses
|
|
to achieve this is to wrap all dependencies within a condition checking the
|
|
toolchain.
|
|
|
|
.. code::
|
|
|
|
group("my_application_images") {
|
|
deps = [] # Empty in the default toolchain.
|
|
|
|
if (current_toolchain != default_toolchain) {
|
|
# This is only evaluated by Pigweed target toolchains, which configure
|
|
# all of the required options to build Pigweed code.
|
|
deps += [ "//images:evt" ]
|
|
}
|
|
}
|
|
|
|
# The images group is instantiated for each of the project's Pigweed targets.
|
|
group("my_pigweed_target") {
|
|
deps = [ ":my_application_images(//toolchains:my_pigweed_target)" ]
|
|
}
|
|
|
|
.. warning::
|
|
Pigweed's default toolchain is never used, so it is set to an empty toolchain
|
|
which doesn't define any tools. ``//BUILD.gn`` contains conditions which check
|
|
that the current toolchain is not the default before declaring any GN target
|
|
dependencies to prevent the default toolchain from evaluating any other BUILD
|
|
files. All GN targets added to the build must be placed under one of these
|
|
conditional scopes.
|
|
|
|
"default" group
|
|
^^^^^^^^^^^^^^^
|
|
The root ``BUILD.gn`` file can define a special group named ``default``. If
|
|
present, Ninja will build this group when invoked without arguments.
|
|
|
|
.. tip::
|
|
Defining a ``default`` group makes using ``pw watch`` simple!
|
|
|
|
Optimization levels
|
|
^^^^^^^^^^^^^^^^^^^
|
|
Pigweed's ``//BUILD.gn`` defines the ``pw_default_optimization_level`` build
|
|
arg, which specifies the optimization level to use for the default groups
|
|
(``host``, ``stm32f429i``, etc.). The supported values for
|
|
``pw_default_optimization_level`` are:
|
|
|
|
* ``debug`` -- create debugging-friendly binaries (``-Og``)
|
|
* ``size_optimized`` -- optimize for size (``-Os``)
|
|
* ``speed_optimized`` -- optimized for speed, without increasing code size
|
|
(``-O2``)
|
|
|
|
Pigweed defines versions of its groups in ``//BUILD.gn`` for each optimization
|
|
level. Rather than relying on ``pw_default_optimization_level``, you may
|
|
directly build a group at the desired optimization level:
|
|
``<group>_<optimization>``. Examples include ``host_clang_debug``,
|
|
``host_gcc_size_optimized``, and ``stm32f429i_speed_optimized``.
|
|
|
|
Upstream GN target groups
|
|
^^^^^^^^^^^^^^^^^^^^^^^^^
|
|
In upstream, Pigweed splits its top-level GN targets into a few logical groups,
|
|
which are described below. In order to build a GN target, it *must* be listed in
|
|
one of the groups in this file.
|
|
|
|
.. important::
|
|
|
|
Pigweed's top-level ``BUILD.gn`` file should not be used by downstream
|
|
projects. Projects that wish to pull all of Pigweed's code into their build
|
|
may use the ``pw_modules`` and ``pw_module_tests`` variables in
|
|
``modules.gni``.
|
|
|
|
apps
|
|
~~~~
|
|
This group defines the application images built in Pigweed. It lists all of the
|
|
common images built across all Pigweed targets, such as modules' example
|
|
executables. Each Pigweed target can additionally provide its own specific
|
|
images through the ``pw_TARGET_APPLICATIONS`` build arg, which is included by
|
|
this group.
|
|
|
|
host_tools
|
|
~~~~~~~~~~
|
|
This group defines host-side tooling binaries built for Pigweed.
|
|
|
|
runtime_sanitizers
|
|
~~~~~~~~~~~~~~~~~~
|
|
This group defines host-side build targets for Clang runtime sanitizers.
|
|
Next runtime sanitizers supported:
|
|
|
|
* ``asan`` -- `AddressSanitizer`_ is a fast memory error detector.
|
|
* ``msan`` -- `MemorySanitizer`_ is a detector of uninitialized reads.
|
|
* ``ubsan`` -- `UndefinedBehaviorSanitizer`_ is a fast undefined behavior detector.
|
|
* ``ubsan_heuristic`` -- `UndefinedBehaviorSanitizer`_ with the following
|
|
additional checks enabled:
|
|
|
|
* ``integer``: Checks for undefined or suspicious integer behavior.
|
|
* ``float-divide-by-zero``: Checks for floating point division by zero.
|
|
* ``implicit-conversion"``: Checks for suspicious behavior of implicit conversions.
|
|
* ``nullability``: Checks for null as function arg, lvalue and return type.
|
|
|
|
These additional checks are heuristic and may not correspond to undefined
|
|
behavior.
|
|
* ``tsan`` -- `ThreadSanitizer`_ is a tool that detects data races.
|
|
|
|
Results of building this group are ``host_clang_<sanitizer>`` build directories
|
|
with ``pw_module_tests`` per supported sanitizer.
|
|
|
|
.. _AddressSanitizer: https://clang.llvm.org/docs/AddressSanitizer.html
|
|
.. _MemorySanitizer: https://clang.llvm.org/docs/MemorySanitizer.html
|
|
.. _UndefinedBehaviorSanitizer: https://clang.llvm.org/docs/UndefinedBehaviorSanitizer.html
|
|
.. _ThreadSanitizer: https://clang.llvm.org/docs/ThreadSanitizer.html
|
|
|
|
pw_modules
|
|
~~~~~~~~~~
|
|
This group lists the main libraries for all of Pigweed's modules.
|
|
|
|
The modules in the ``pw_modules`` group are listed in the ``pw_modules``
|
|
variable, which is provided by ``modules.gni``.
|
|
|
|
pw_module_tests
|
|
~~~~~~~~~~~~~~~
|
|
All modules' unit tests are collected here, so that they can all be run at once.
|
|
|
|
The test groups in ``pw_module_tests`` group are listed in the
|
|
``pw_module_tests`` variable, which is provided by ``modules.gni``.
|
|
|
|
pigweed_default
|
|
~~~~~~~~~~~~~~~
|
|
This group defines everything built in a Pigweed build invocation by collecting
|
|
the above groups and conditionally depending on them based on the active Pigweed
|
|
target's configuration. Generally, new dependencies should not be added here;
|
|
instead, use one of the groups listed above.
|
|
|
|
The ``pigweed_default`` group is instantiated for each upstream Pigweed target's
|
|
toolchain.
|
|
|
|
Pigweed target instantiations
|
|
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
|
|
These groups wrap ``pigweed_default`` with a specific target toolchain. They are
|
|
named after the Pigweed target, e.g. ``host_clang``, ``stm32f429i``, etc.
|
|
|
|
Other BUILD files: //\*\*/BUILD.gn
|
|
----------------------------------
|
|
The rest of the ``BUILD.gn`` files in the tree define libraries, configs, and
|
|
build args for each of the modules in a Pigweed project.
|
|
|
|
Project configuration: //build_overrides/pigweed.gni
|
|
----------------------------------------------------
|
|
Each Pigweed project must contain a Pigweed configuration file at a known
|
|
location in the GN build tree. Currently, this file only contains a single build
|
|
argument, which must be set to the GN build path to the root of the Pigweed
|
|
repository within the project.
|
|
|
|
Module variables
|
|
----------------
|
|
As Pigweed is intended to be a subcomponent of a larger project, it cannot assume
|
|
where it or its modules is located. Therefore, Pigweed's upstream BUILD.gn files
|
|
do not use absolute paths; instead, variables are defined pointing to each of
|
|
Pigweed's modules, set relative to a project-specific ``dir_pigweed``.
|
|
|
|
To depend on Pigweed modules from GN code, import Pigweed's overrides file and
|
|
reference these module variables.
|
|
|
|
.. code::
|
|
|
|
# This must be imported before .gni files from any other Pigweed modules. To
|
|
# prevent gn format from reordering this import, it must be separated by a
|
|
# blank line from other imports.
|
|
|
|
import("//build_overrides/pigweed.gni")
|
|
|
|
GN target type wrappers
|
|
-----------------------
|
|
To facilitate injecting global configuration options, Pigweed defines wrappers
|
|
around builtin GN target types such as ``source_set`` and ``executable``. These
|
|
are defined within ``$dir_pw_build/target_types.gni``.
|
|
|
|
.. note::
|
|
To take advantage of Pigweed's flexible target configuration system, use
|
|
``pw_*`` target types (e.g. ``pw_source_set``) in your BUILD.gn files instead
|
|
of GN builtins.
|
|
|
|
Pigweed targets
|
|
---------------
|
|
To build for a specific hardware platform, builds define Pigweed targets. These
|
|
are essentially GN toolchains which set special arguments telling Pigweed how to
|
|
build. For information on Pigweed's target system, refer to
|
|
:ref:`docs-targets`.
|
|
|
|
The empty toolchain
|
|
-------------------
|
|
Pigweed's ``BUILDCONFIG.gn`` sets the project's default toolchain to a "empty"
|
|
toolchain which does not specify any compilers or override any build arguments.
|
|
Downstream projects are recommended to do the same, following the steps
|
|
described in :ref:`top-level-build` to configure builds for each of their
|
|
Pigweed targets.
|
|
|
|
.. admonition:: Why use an empty toolchain?
|
|
|
|
To support some of its advanced (and useful!) build features, Pigweed requires
|
|
the ability to generate new toolchains on the fly. This requires having
|
|
knowledge of the full configuration of the current toolchain (which is easy if
|
|
it's all defined within a scope), something which is impractical to achieve
|
|
using the default toolchain.
|
|
|
|
Additionally, there are some cases where GN treats default and non-default
|
|
toolchains differently. By not using the default toolchain, we avoid having
|
|
to deal with these inconsistencies.
|
|
|
|
It is possible to build Pigweed using only the default toolchain, but it
|
|
requires a more complicated setup to get everything working and should be
|
|
avoided unless necessary (for example, when integrating with a large existing
|
|
GN-based project).
|
|
|
|
Upstream development examples
|
|
-----------------------------
|
|
If developing for upstream Pigweed, some common build use cases are described
|
|
below.
|
|
|
|
Building a custom executable/app image
|
|
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
|
|
|
|
1. Define your executable GN target using the ``pw_executable`` template.
|
|
|
|
.. code::
|
|
|
|
# //foo/BUILD.gn
|
|
pw_executable("foo") {
|
|
sources = [ "main.cc" ]
|
|
deps = [ ":libfoo" ]
|
|
}
|
|
|
|
2. In the root ``BUILD.gn`` file, add the executable's GN target to the ``apps``
|
|
group.
|
|
|
|
.. code::
|
|
|
|
# //BUILD.gn
|
|
group("apps") {
|
|
deps = [
|
|
# ...
|
|
"//foo", # Shorthand for //foo:foo
|
|
]
|
|
}
|
|
|
|
3. Run the ninja build to compile your executable. The apps group is built by
|
|
default, so there's no need to provide a target. The executable will be
|
|
compiled for every supported Pigweed target.
|
|
|
|
.. code::
|
|
|
|
ninja -C out
|
|
|
|
Alternatively, build your executable by itself by specifying its path to
|
|
Ninja. When building a GN target manually, the Pigweed target for which it
|
|
is built must be specified on the Ninja command line.
|
|
|
|
For example, to build for the Pigweed target ``host_gcc_debug``:
|
|
|
|
.. code::
|
|
|
|
ninja -C out host_gcc_debug/obj/foo/bin/foo
|
|
|
|
.. note::
|
|
|
|
The path passed to Ninja is a filesystem path within the ``out`` directory,
|
|
rather than a GN path. This path can be found by running ``gn outputs``.
|
|
|
|
4. Retrieve your compiled binary from the out directory. It is located at the
|
|
path
|
|
|
|
.. code::
|
|
|
|
out/<pw_target>/obj/<gn_path>/{bin,test}/<executable>
|
|
|
|
where ``pw_target`` is the Pigweed target for which the binary was built,
|
|
``gn_path`` is the GN path to the BUILD.gn file defining the executable,
|
|
and ``executable`` is the executable's GN target name (potentially with an
|
|
extension). Note that the executable is located within a ``bin`` subdirectory
|
|
in the module (or ``test`` for unit tests defined with ``pw_test``).
|
|
|
|
For example, the ``foo`` executable defined above and compiled for the
|
|
Pigweed target stm32f429i_disc1_debug is found at:
|
|
|
|
.. code::
|
|
|
|
out/stm32f429i_disc1_debug/obj/foo/bin/foo
|
|
|
|
CMake
|
|
-----
|
|
A well-known name in C/C++ development, `CMake`_ is widely used by all kinds of
|
|
projects, including embedded devices. Pigweed's CMake support is provided
|
|
primarily for projects that have an existing CMake build and wish to integrate
|
|
Pigweed modules.
|
|
|
|
Bazel
|
|
-----
|
|
The open source version of Google's internal build system. `Bazel`_ has been
|
|
growing in popularity within the open source world, as well as being adopted by
|
|
various proprietary projects. Its modular structure makes it a great fit for
|
|
à la carte usage.
|
|
|
|
.. note::
|
|
Bazel support is experimental and only for the brave for now. If you are
|
|
looking for stable set of build API's please use GN.
|
|
|
|
The Bazel build
|
|
===============
|
|
This section describes Pigweed's Bazel build structure, how it is used upstream,
|
|
build conventions, and recommendations for Pigweed-based projects. While
|
|
containing some details about how Bazel works in general, this section is not
|
|
intended to be a guide on how to use Bazel. To learn more about the tool itself,
|
|
refer to the official `Bazel reference`_.
|
|
|
|
.. _Bazel reference: https://www.bazel.build/
|
|
|
|
General usage
|
|
-------------
|
|
While described in more detail in the Bazel docs there a few Bazel features that
|
|
are of particular importance when targeting embedded platforms. The most
|
|
commonly used commands used in bazel are;
|
|
|
|
.. code:: sh
|
|
|
|
bazel build //your:target
|
|
bazel test //your:target
|
|
bazel coverage //your:target
|
|
|
|
.. note:: Code coverage support is only available on the host for now.
|
|
|
|
Building
|
|
^^^^^^^^
|
|
When it comes to building/testing your Bazel target for a specific Pigweed
|
|
target (e.g. stm32f429i-discovery) a slight variation is required.
|
|
|
|
.. code:: sh
|
|
|
|
bazel build //your:target \
|
|
--platforms=@pigweed//pw_build/platforms:stm32f429i-disc1
|
|
|
|
For more information on how to create your own platforms refer to the official
|
|
`Bazel platforms reference`_. You may also find helpful examples of constraints
|
|
and platforms in the '//pw_build/platforms' and '//pw_build/constraints'
|
|
directories.
|
|
|
|
.. _Bazel platforms reference: https://docs.bazel.build/versions/main/platforms.html
|
|
|
|
Testing
|
|
^^^^^^^
|
|
Running tests on an embedded target with Bazel is possible although support for
|
|
this is experimental. The easiest way of achieving this at the moment is to use
|
|
Bazel's '--run_under' flag. To make this work create a Bazel target
|
|
('//your_handler') that;
|
|
|
|
1. Takes a single argument (the path to the elf) and uploads the elf to your
|
|
Pigweed target.
|
|
2. Connects with your target using serial or other communication method.
|
|
3. Listens to the communication transport for the keywords ("PASSED", "FAIL")
|
|
and returns (0, 1) respectively if one of the keywords is intercepted. (This
|
|
step assumes you are using the pw_unit_test package and it is configured for
|
|
your target).
|
|
4. Run;
|
|
|
|
.. code:: sh
|
|
|
|
bazel test //your:test --platforms=//your/platform --run_under=//your_handler
|
|
|
|
Code Coverage
|
|
^^^^^^^^^^^^^
|
|
Making use of the code coverage functionality in Bazel is straightforward.
|
|
|
|
1. Add the following lines to your '.bazelrc'.
|
|
|
|
.. code:: sh
|
|
|
|
coverage --experimental_generate_llvm_lcov
|
|
coverage --combined_report=lcov
|
|
|
|
2. Generate a combined lcov coverage report. This will produce a combined lcov
|
|
coverage report at the path 'bazel-out/_coverage/_coverage_report.dat'. e.g.
|
|
|
|
.. code:: sh
|
|
|
|
bazel coverage //pw_log/...
|
|
|
|
3. View the results using the command line utility 'lcov'.
|
|
|
|
.. code:: sh
|
|
|
|
lcov --list bazel-out/_coverage/_coverage_report.dat
|
|
|
|
Configuration
|
|
-------------
|
|
Generally speaking there are three primary concepts that make up Bazel's
|
|
configuration API.
|
|
|
|
1. Selects
|
|
2. Compatibility lists
|
|
3. Flags/Build settings
|
|
|
|
Selects
|
|
^^^^^^^
|
|
Selects are useful for specifying different dependencies/source depending on the
|
|
platform that is currently being targeted. For more information on this please
|
|
see the `Bazel selects reference`_. e.g.
|
|
|
|
.. code:: py
|
|
|
|
pw_cc_library(
|
|
name = "some_platform_dependant_library",
|
|
deps = select({
|
|
"@platforms//cpu:armv7e-m": [":arm_libs"],
|
|
"//conditions:default": [":host_libs"],
|
|
}),
|
|
)
|
|
|
|
Compatibility lists
|
|
^^^^^^^^^^^^^^^^^^^
|
|
Compatibility lists allow you to specify which platforms your targets are
|
|
compatible with. Consider an example where you want to specify that a target is
|
|
compatible with only a host os;
|
|
|
|
.. code:: py
|
|
|
|
pw_cc_library(
|
|
name = "some_host_only_lib",
|
|
srcs = ["host.cc"],
|
|
target_compatible_with = select({
|
|
"@platforms//os:windows": [],
|
|
"@platforms//os:linux": [],
|
|
"@platforms//os:macos": [],
|
|
"//conditions:default": ["@platforms//:incompatible"],
|
|
}),
|
|
)
|
|
|
|
In this case building from or for either Windows/Linux/Mac will be supported, but
|
|
other OS's will fail if this target is explicitly depended on. However if
|
|
building with a wild card for a non-host platform this target will be skipped
|
|
and the build will continue. e.g.
|
|
|
|
.. code:: sh
|
|
|
|
bazel build //... --platforms=@pigweed//pw_build/platforms:cortex_m0
|
|
|
|
This allows for you to easily create compatibility matricies without adversely
|
|
affecting your ability build your entire repo for a given Pigweed target.
|
|
For more detailed information on how to use the target_compatible_with attribute
|
|
please see `Bazel target_compatible_with reference`_.
|
|
|
|
Flags/build settings
|
|
^^^^^^^^^^^^^^^^^^^^
|
|
Flags/build settings are particularly useful in scenarios where you may want to
|
|
be able to quickly inject a dependency from the command line but don't
|
|
necessarily want to create an entirely new set of constraints to use with a
|
|
select statement.
|
|
|
|
.. note::
|
|
The scope for what is possible with build flags/settings goes well beyond
|
|
what will be described here. For more detailed information on flags/settings
|
|
please see `Bazel config reference`_.
|
|
|
|
A simple example of when it is useful to use a label_flag is when you want to
|
|
swap out a single dependency from the command line. e.g.
|
|
|
|
.. code:: py
|
|
|
|
pw_cc_library(
|
|
name = "some_default_io",
|
|
srcs = ["default_io.cc"],
|
|
)
|
|
|
|
pw_cc_library(
|
|
name = "some_other_io",
|
|
srcs = ["other_io.cc"],
|
|
)
|
|
|
|
label_flag(
|
|
name = "io",
|
|
default_build_setting = ":some_default_io",
|
|
)
|
|
|
|
pw_cc_library(
|
|
name = "some_target_that_needs_io",
|
|
deps = [":io"],
|
|
)
|
|
|
|
From here the label_flag by default redirects to the target ":some_default_io",
|
|
however it is possible to override this from the command line. e.g.
|
|
|
|
.. code:: sh
|
|
|
|
bazel build //:some_target_that_needs_io --//:io=//:some_other_io
|
|
|
|
|
|
|
|
.. _Bazel selects reference: https://docs.bazel.build/versions/main/configurable-attributes.html#select-and-dependencies
|
|
|
|
.. _Bazel target_compatible_with reference: https://docs.bazel.build/versions/main/platforms.html#skipping-incompatible-targets
|
|
|
|
.. _Bazel config reference: https://docs.bazel.build/versions/main/skylark/config.html
|
|
|
|
|
|
Pigweeds configuration
|
|
^^^^^^^^^^^^^^^^^^^^^^
|
|
Pigweeds Bazel configuration API is designed to be distributed across the
|
|
Pigweed repository and/or your downstream repository. If you are coming from
|
|
GN's centralized configuration API it might be useful to think about
|
|
Pigweed+Bazel's configuration as the transpose of GN's configuration. The
|
|
configuration interface that is supported by Pigweed is designed to start simple
|
|
and then grow with your project.
|
|
|
|
.. note::
|
|
There are plans to extend the configuration API for Bazel. However,
|
|
currently the only configurations that are available under the Bazel+Pigweed
|
|
configuration API is the ability to switch facade backends. For more
|
|
information on what this is please see the
|
|
:ref:`docs-module-structure-facades` section of :ref:`docs-module-structure`.
|
|
|
|
Consider a scenario that you are building a flight controller for a
|
|
spacecraft. But have very little experience with Pigweed and you have just
|
|
landed here. First things first you would;
|
|
|
|
1. Set up your WORKSPACE to fetch the Pigweeds repository. Then add the
|
|
dependencies that you need from Pigweeds WORKSPACE.
|
|
|
|
2. Add a pigweed_config rule to your WORKSPACE, using Pigweed's default
|
|
configuration.
|
|
|
|
.. code:: py
|
|
|
|
# WORKSPACE ...
|
|
load("//pw_build:target_config.bzl", "pigweed_config")
|
|
|
|
# Configure Pigweeds backend.
|
|
pigweed_config(
|
|
name = "pigweed_config",
|
|
build_file = "@pigweed//targets:default_config.BUILD",
|
|
)
|
|
|
|
.. note::
|
|
We are aware, that the experience of setting up your WORKSPACE file to work
|
|
with pigweed is less than ideal. This API is under construction, but we are
|
|
working on this!
|
|
|
|
..
|
|
TODO: Add in a better way to pull down WORKSPACE dependencies in Bazel then
|
|
add docs in here.
|
|
|
|
Now to explain what is going on here. Housed under the "pigweed_config" remote
|
|
repository is a set of configuration flags. These can be used to inject
|
|
dependencies into facades to override the default backend.
|
|
|
|
Continuing on with our scenario, consider that you maybe want to try using the
|
|
'//pw_chrono' module. So you create a target in your repository like so;
|
|
|
|
.. code::
|
|
|
|
# BUILD
|
|
pw_cc_library(
|
|
name = "time_is_relative",
|
|
srcs = ["relative_time_on_earth.cc"],
|
|
deps = ["@pigweed//pw_chrono"],
|
|
)
|
|
|
|
Now this should work out of the box for any host operating system. e.g. Running;
|
|
|
|
.. code::
|
|
|
|
bazel build //:time_is_relative
|
|
|
|
will produce a working library. But as your probably here because Pigweed offers
|
|
a set of embedded libraries you might be interested in running your code on some
|
|
random micro-controller/FPGA combined with an RTOS. For now let's assume that by
|
|
some coincidence you are using FreeRTOS and are happy to make use
|
|
of our default '//pw_chrono' backend for FreeRTOS. You could build the following
|
|
with;
|
|
|
|
.. code:: sh
|
|
|
|
bazel build //:time_is_relative \
|
|
--platforms=@pigweed//pw_build/platforms:freertos
|
|
|
|
There is a fair bit to unpack here in terms of how our configuration system
|
|
is determining which dependencies to choose for your build. The dependency
|
|
tree (that is important for configuration) in a project such as this would
|
|
look like.
|
|
|
|
.. code::
|
|
|
|
@pigweed//pw_chrono:pw_chrono_facade <-----------.
|
|
^ |
|
|
| @pigweed//pw_chrono_freertos:system_clock
|
|
| (Actual backend)
|
|
| ^
|
|
| |
|
|
| @pigweed//pw_chrono:system_clock_backend_multiplexer
|
|
| Select backend based on OS:
|
|
| [FreeRTOS (X), Embos ( ), STL ( ), Threadx ( )]
|
|
| ^
|
|
| |
|
|
@pigweed//pw_chrono -------> @pigweed_config//:pw_chrono_system_clock_backend
|
|
^ (Injectable)
|
|
|
|
|
//:time_is_relative
|
|
|
|
So when evaluating this setup Bazel checks the dependencies for '//pw_chrono'
|
|
and finds that it depends on "@pigweed_config//:pw_chrono_system_clock_backend" which looks
|
|
like this;
|
|
|
|
.. code:: py
|
|
|
|
# pw_chrono config.
|
|
label_flag(
|
|
name = "pw_chrono_system_clock_backend",
|
|
build_setting_default = "@pigweed//pw_chrono:system_clock_backend_multiplexer",
|
|
)
|
|
|
|
Looking at the 'build_setting_default' we can see that by default it depends
|
|
back on the target "@pigweed//pw_chrono:system_clock_backend_multiplexer". If
|
|
you only had one backend you could actually just change the
|
|
'build_setting_default' to point directly to your backend. However because we
|
|
have four different backends we have to use the select semantics to choose the
|
|
right one. In this case it looks like;
|
|
|
|
.. code:: py
|
|
|
|
pw_cc_library(
|
|
name = "system_clock_backend_multiplexer",
|
|
visibility = ["@pigweed_config//:__pkg__"],
|
|
deps = select({
|
|
"@pigweed//pw_build/constraints/rtos:freertos":
|
|
["//pw_chrono_freertos:system_clock"],
|
|
"@pigweed//pw_build/constraints/rtos:embos":
|
|
["//pw_chrono_embos:system_clock"],
|
|
"@pigweed//pw_build/constraints/rtos:threadx":
|
|
["//pw_chrono_threadx:system_clock"],
|
|
"//conditions:default": ["//pw_chrono_stl:system_clock"],
|
|
}),
|
|
)
|
|
|
|
Intuitively you can see that the first option was selected, which terminates
|
|
the configuration chain.
|
|
|
|
Continuing on with our scenario let's say that you have read
|
|
:ref:`docs-module-structure` and now want to implement your own backend for
|
|
'//pw_chrono' using a hardware RTC. In this case you would create a new
|
|
directory 'pw_chrono_my_hardware_rtc'. To ensure that your new backend compiles
|
|
with the facade an easy and temporary way to override the dependency tree is
|
|
to override the label flag in '@pigweed_config'. For example;
|
|
|
|
.. code:: sh
|
|
|
|
bazel build //:time_is_relative \
|
|
--@pigweed_config//pw_chrono_system_clock_backend=//pw_chrono_my_hardware_rtc:system_clock
|
|
|
|
This temporarily modifies the build graph to look something like this;
|
|
|
|
.. code::
|
|
|
|
@pigweed//pw_chrono:pw_chrono_facade <-----.
|
|
^ |
|
|
| @your_workspace//pw_chrono_my_hardware_rtc:system_clock
|
|
| (Actual backend)
|
|
| ^
|
|
| |
|
|
@pigweed//pw_chrono -> @pigweed_config//:pw_chrono_system_clock_backend
|
|
^ (Injectable)
|
|
|
|
|
//:time_is_relative
|
|
|
|
Now while this is a nice temporary change, but you might find yourself in need
|
|
of a more permanent configuration. Particularly if you want to override multiple
|
|
different backends. In other words if you had several backends to override, that
|
|
would translate to several different command line flags (one for each override).
|
|
This problem further compounds as you have multiple Pigweed targets all
|
|
requiring different combinations of different backends as you can't even reuse
|
|
your command line entries. Instead you would have to memorize the correct
|
|
combination of backends for each of your targets.
|
|
|
|
So continuing on with our scenario, let's say we add a backup micro-controller,
|
|
to our spacecraft. But this backup computer doesn't have a hardware RTC. We
|
|
still want to share the bulk of the code between the two computers but now we
|
|
need two separate implementations for our pw_chrono facade. Let's say we choose
|
|
to keep the primary flight computer using the hardware RTC and switch the backup
|
|
computer over to use Pigweeds default FreeRTOS backend. In this case we might,
|
|
want to do something similar to
|
|
'@pigweed//pw_chrono:system_clock_backend_multiplexer' and create selectable
|
|
dependencies for the two different computers. Now because there are no default
|
|
constraint_setting's that meet our requirements we are going to have to;
|
|
|
|
1. Create a constraint_setting and a set of constraint_value's for the flight
|
|
computer. For example;
|
|
|
|
.. code:: py
|
|
|
|
# //platforms/flight_computer/BUILD
|
|
constraint_setting(
|
|
name = "flight_computer",
|
|
)
|
|
|
|
constraint_value(
|
|
name = "primary",
|
|
constraint_setting = ":flight_computer",
|
|
)
|
|
|
|
constraint_value(
|
|
name = "backup",
|
|
constraint_setting = ":flight_computer",
|
|
)
|
|
|
|
2. Create a set of platforms that can be used to switch constraint_value's.
|
|
For example;
|
|
|
|
.. code:: py
|
|
|
|
# //platforms/BUILD
|
|
platform(
|
|
name = "primary_computer",
|
|
constraint_values = ["//platforms/flight_computer:primary"],
|
|
)
|
|
|
|
platform(
|
|
name = "backup_computer",
|
|
constraint_values = ["//platforms/flight_computer:backup"],
|
|
)
|
|
|
|
3. Create a target multiplexer that will select the right backend depending on
|
|
which computer you are using. For example;
|
|
|
|
.. code:: py
|
|
|
|
# //pw_chrono/BUILD
|
|
load("//pw_build:pigweed.bzl", "pw_cc_library")
|
|
|
|
pw_cc_library(
|
|
name = "system_clock_backend_multiplexer",
|
|
deps = select({
|
|
"//platforms/flight_computer:primary": [
|
|
"//pw_chrono_my_hardware_rtc:system_clock",
|
|
],
|
|
"//platforms/flight_computer:backup": [
|
|
"@pigweed//pw_chrono_freertos:system_clock",
|
|
],
|
|
"//conditions:default": [
|
|
"@pigweed//pw_chrono_stl:system_clock",
|
|
],
|
|
}),
|
|
)
|
|
|
|
4. Copy and paste across the target/default_config.BUILD across from the
|
|
Pigweed repository and modifying the build_setting_default for the target
|
|
'pw_chrono_system_clock_backend' to point to your new system_clock_backend_multiplexer
|
|
target. For example;
|
|
|
|
This;
|
|
|
|
.. code:: py
|
|
|
|
# @pigweed//target:default_config.BUILD
|
|
label_flag(
|
|
name = "pw_chrono_system_clock_backend",
|
|
build_setting_default = "@pigweed//pw_chrono:system_clock_backend_multiplexer",
|
|
)
|
|
|
|
Becomes this;
|
|
|
|
.. code:: py
|
|
|
|
# @your_workspace//target:your_config.BUILD
|
|
label_flag(
|
|
name = "pw_chrono_system_clock_backend",
|
|
build_setting_default =
|
|
"@your_workspace//pw_chrono:system_clock_backend_multiplexer",
|
|
)
|
|
|
|
5. Switch your workspace 'pigweed_config' rule over to use your custom config.
|
|
|
|
.. code:: py
|
|
|
|
# WORKSPACE
|
|
pigweed_config(
|
|
name = "pigweed_config",
|
|
build_file = "//target/your_config.BUILD",
|
|
)
|
|
|
|
Building your target now will result in slightly different build graph. For
|
|
example, running;
|
|
|
|
.. code:: sh
|
|
|
|
bazel build //:time_is_relative --platforms=//platforms:primary_computer
|
|
|
|
Will result in a build graph that looks like;
|
|
|
|
.. code::
|
|
|
|
@pigweed//pw_chrono:pw_chrono_facade <---.
|
|
^ |
|
|
| @your_workspace//pw_chrono_my_hardware_rtc:system_clock
|
|
| (Actual backend)
|
|
| ^
|
|
| |
|
|
| @your_workspace//pw_chrono:system_clock_backend_multiplexer
|
|
| Select backend based on OS:
|
|
| [Primary (X), Backup ( ), Host only default ( )]
|
|
| ^
|
|
| |
|
|
@pigweed//pw_chrono -> @pigweed_config//:pw_chrono_system_clock_backend
|
|
^ (Injectable)
|
|
|
|
|
//:time_is_relative
|