5.0 KiB

Raw Blame History

test_droid: Quick Primer

References

Objective

This document contains instructions for Brillo/Android developers interested in running automated integration tests at their desk.

Usage

The autotest repository, the test_droid tool, and all tests are checked out in both AOSP and internal Android trees at external/autotest. You need a copy of the autotest source code and an adb binary in your PATH to run tests.

Running tests against a single local device under test

Once you have a local copy of the autotest source, you can easily run tests against a DUT connected directly to your workstation via a USB cable. Please note your first time running test_droid it will download and install a number of required packages locally into your autotest checkout.

Run site_utils/test_droid.py from your autotest checkout to launch a test against a given DUT:

 $ ./site_utils/test_droid.py <Test Name>

For example, to run the brillo_WhitelistedGtests test:

 $ ./site_utils/test_droid.py brillo_WhitelistedGtests

test_droid can run multiple tests at once:

 $ ./site_utils/test_droid.py brillo_WhitelistedGtests brillo_KernelVersionTest

As well as test suites:

 $ ./site_utils/test_droid.py suite:brillo-bvt

Selecting a specific device

If you have more than one device connected, you'll have to specify its serial number. First, look it up:

 $ adb devices
* daemon started successfully *
List of devices attached
7d52318 device

Then use it when running:

 $ ./site_utils/test_droid.py -s 7d52318 brillo_WhitelistedGtests

Running tests that require multiple devices under test

Autotest now supports the concept of testbeds, which are multiple devices being controlled by a single test. test_droid supports running these tests by specifying a comma separated list of serials as the test device:

 $ adb devices
List of devices attached
emulator-5554 device
7d52318 device

 $ ./site_utils/test_droid.py -s emulator-5554,7d52318 testbed_DummyTest

Running tests against a remote device under test

test_droid can run tests against devices connected to a remote server. This requires passwordless SSH access from the workstation to the remote server. If no username is specified, test_droid will try the root and adb users. If using the adb user, make sure it has passwordless sudo rights to run the adb and fastboot commands. You can specify a different user in the remote host name (the same passwordless requirement applies).

The easiest way to set this up is to use the Chrome OS testing keys. Add to your SSH config an entry that looks like the following:

HostName <Remote Server IP or Hostname>
  Port 9222
  User root
  CheckHostIP no
  StrictHostKeyChecking no
  IdentityFile ~/.ssh/testing_rsa
  Protocol 2

To run the test:

 $ ./site_utils/test_droid.py \
       -r <Remote Server IP or Hostname> \
       <Test Name>

 $ ./site_utils/test_droid.py \
       -r <User>@<Remote Server IP or Hostname> \
       <Test Name>

 $ ./site_utils/test_droid.py -r 100.96.48.119 suite:brillo-bvt

Advanced: Uploading Commits for Review

Currently Autotest in AOSP is read-only, so you cannot use repo upload to upload code changes. If you do edit or add a new test, make a commit and upload it to https://chromium-review.googlesource.com.

Due to the way Android does its automerging, the AOSP mirror of Autotest cannot be uploaded directly back to the Chromium upstream because the upstream does not contain the merge commits. It will also reject the automerge committer ID. To work around this, if you need to upload a commit to Autotest, you first have to fetch and check out the upstream branch before making your changes:

 $ git remote add -t master cros \
     https://chromium.googlesource.com/chromiumos/third_party/autotest
 $ git fetch cros
 $ git checkout cros/master
 $ git checkout -b <local branch name>
 $ git branch --set-upstream-to=cros/master

Be sure to run pylint on every file you touch:

 $ ./utils/run_pylint.py <file name>

Run autotest unittests (which usually requires external packages):

 $ utils/build_externals.py  # install any missing os packages with apt-get
 $ utils/unittest_suite.py

Then upload your commit for review:

 $ git push cros <local branch name>:refs/for/master

Limitations

Testing on Brillo/Android (and test_droid by extension) is currently limited to server-side tests, which run on an autotest server and control a Brillo/Android DUT (device under test) via remote command execution. In the context of test_droid, the test logic is running on your development machine, and controlling devices with commands executed via adb.

test_droid does not support the autoupdate end-to-end test. For instructions on how to run this test please refer to the Running Brillo/Android Autoupdate End-to-End Test doc.

5.0 KiB Raw Blame History