GCLOUD_FIREBASE_TEST_ANDROID_RUN(1)

NAME

gcloud firebase test android run - invoke a test in Firebase Test Lab for Android and view test results

SYNOPSIS

gcloud firebase test android run [ARGSPEC] [--app=APP] [--device=DIMENSION=VALUE,[DIMENSION=VALUE,...]] [--test=TEST] [--timeout=TIMEOUT] [--type=TYPE] [--app-package=APP_PACKAGE] [--async] [--no-auto-google-login] [--directories-to-pull=[DIR_TO_PULL,...]] [--environment-variables=[KEY=VALUE,...]] [--obb-files=OBB_FILE,[OBB_FILE,...]] [--results-bucket=RESULTS_BUCKET] [--results-dir=RESULTS_DIR] [--results-history-name=RESULTS_HISTORY_NAME] [--test-package=TEST_PACKAGE] [--test-runner-class=TEST_RUNNER_CLASS] [--test-targets=TEST_TARGET,[TEST_TARGET,...]] [--app-initial-activity=APP_INITIAL_ACTIVITY] [--max-depth=int] [--max-steps=int] [--robo-directives=[TYPE:RESOURCE_NAME=INPUT,...]] [--device-ids=MODEL_ID,[MODEL_ID,...], -d MODEL_ID,[MODEL_ID,...]] [--locales=LOCALE,[LOCALE,...], -l LOCALE,[LOCALE,...]] [--orientations=ORIENTATION,[ORIENTATION,...], -o ORIENTATION,[ORIENTATION,...]] [--os-version-ids=OS_VERSION_ID,[OS_VERSION_ID,...], -v OS_VERSION_ID,[OS_VERSION_ID,...]] [--filter=EXPRESSION] [--limit=LIMIT] [--page-size=PAGE_SIZE] [--sort-by=[FIELD,...]] [--uri] [GCLOUD_WIDE_FLAG ...]

DESCRIPTION
gcloud firebase test android run invokes and monitors tests in Firebase Test Lab for Android. Two main types of tests are currently supported:

    •   robo: runs a smart, automated exploration of the activities in your Android app which records any installation failures or crashes and builds an activity map with associated screenshots and video.

    •   instrumentation: runs automated unit or integration tests written using a testing framework. Firebase Test Lab for Android currently supports the Espresso, Robotium and UI Automator 2.0 testing frameworks.

The type of test to run can be specified with the --type flag, although the type can often be inferred from other flags. Specifically, if the --test flag is present, the test --type will default to instrumentation. If --test is not present, then --type defaults to robo. All arguments for gcloud firebase test android run may be specified on the command line and/or within an argument file. Run $ gcloud topic arg-files for more information about argument files.

POSITIONAL ARGUMENTS

    [ARGSPEC]
        An ARG_FILE:ARG_GROUP_NAME pair, where ARG_FILE is the path to a file containing groups of test arguments in yaml format, and ARG_GROUP_NAME is the particular yaml object holding a group of arg:value pairs to use. Run $ gcloud topic arg-files for more information and examples.

COMMONLY USED FLAGS

    --app=APP
        The path to the application binary file. The path may be in the local filesystem or in Google Cloud Storage using gs:// notation.

    --device=DIMENSION=VALUE,[DIMENSION=VALUE,...]
        A list of DIMENSION=VALUE pairs which specify a target device to test against. This flag may be repeated to specify multiple devices. The four device dimensions are: model, version, locale, and orientation. If any dimensions are omitted, they will use a default value. The default value can be found with the list command for each dimension, $ gcloud firebase test android <dimension> list. --device is now the preferred way to specify test devices and may not be used in conjunction with --devices-ids, --os-version-ids, --locales, or --orientations. Omitting all of the preceding dimension-related flags will run tests against a single device using defaults for all four device dimensions. Examples:
        --device model=Nexus6 --device version=23,orientation=portrait --device model=shamu,version=22,locale=zh_CN,orientation=landscape

    --test=TEST
        The path to the binary file containing instrumentation tests. The given path may be in the local filesystem or in Google Cloud Storage using gs:// notation.

    --timeout=TIMEOUT
        The max time this test execution can run before it is cancelled (default: 15m). It does not include any time necessary to prepare and clean up the target device. The maximum possible testing time is 30m on physical devices and 60m on virtual devices. The TIMEOUT units can be h, m, or s. If no unit is given, seconds are assumed. Examples:

        —   --timeout 1h is 1 hour

        —   --timeout 5m is 5 minutes

        —   --timeout 200s is 200 seconds

        —   --timeout 100 is 100 seconds

    --type=TYPE
        The type of test to run. TYPE must be one of: instrumentation, robo.

FLAGS

    --app-package=APP_PACKAGE
        The Java package of the application under test (default: extracted from the APK manifest).

    --async
        Invoke a test asynchronously without waiting for test results.

    --auto-google-login
        Automatically log into the test device using a preconfigured Google account before beginning the test. Enabled by default, use --no-auto-google-login to disable.

    --directories-to-pull=[DIR_TO_PULL,...]
        A list of paths that will be copied from the device's storage to the designated results bucket after the test is complete. (For example --directories-to-pull /sdcard/tempDir1,/data/tempDir2)

    --environment-variables=[KEY=VALUE,...]
        A comma-separated, key=value, map of environment variables and their desired values. The environment variables passed here will be mirrored on to the adb run command. For example, specify --environment-variables coverage=true,coverageFile="/sdcard/tempDir/coverage.ec" to enable code coverage and provide a file path to store the coverage results.

    --obb-files=OBB_FILE,[OBB_FILE,...]
        A list of one or two Android OBB file names which will be copied to each test device before the tests will run (default: None). Each OBB file name must conform to the format as specified by Android (e.g. [main|patch].0300110.com.example.android.obb) and will be installed into <shared-storage>/Android/obb/<package-name>/ on the test device.

    --results-bucket=RESULTS_BUCKET
        The name of a Google Cloud Storage bucket where raw test results will be stored (default: "test-lab-<random-UUID>"). Note that the bucket must be owned by a billing-enabled project, and that using a non-default bucket will result in billing charges for the storage used.

    --results-dir=RESULTS_DIR
        The name of a unique Google Cloud Storage object within the results bucket where raw test results will be stored (default: a timestamp with a random suffix). Caution: if specified, this argument must be unique for each test matrix you create, otherwise results from multiple test matrices will be overwritten or intermingled.

    --results-history-name=RESULTS_HISTORY_NAME
        The history name for your test results (an arbitrary string label; default: the application's label from the APK manifest). All tests which use the same history name will have their results grouped together in the Firebase console in a time-ordered test history list.

ANDROID INSTRUMENTATION TEST FLAGS

    --test-package=TEST_PACKAGE
        The Java package name of the instrumentation test (default: extracted from the APK manifest).

    --test-runner-class=TEST_RUNNER_CLASS
        The fully-qualified Java class name of the instrumentation test runner (default: the last name extracted from the APK manifest).

    --test-targets=TEST_TARGET,[TEST_TARGET,...]
        A list of one or more instrumentation test targets to be run (default: all targets). Each target must be fully qualified with the package name or class name, in one of these formats:

        —   "package package_name"

        —   "class package_name.class_name"

        —   "class package_name.class_name#method_name".

ANDROID ROBO TEST FLAGS

    --app-initial-activity=APP_INITIAL_ACTIVITY
        The initial activity used to start the app during a robo test.

    --max-depth=int
        The maximum depth of the traversal stack a robo test can explore. Needs to be at least 2 to make Robo explore the app beyond the first activity (default: 50).

    --max-steps=int
        The maximum number of steps/actions a robo test can execute (default: no limit).

    --robo-directives=[TYPE:RESOURCE_NAME=INPUT,...]
        A comma-separated (<type>:<key>=<value>) map of robo_directives that you can use to customize the behavior of Robo test. The type specifies the action type of the directive, which may take on values click or text. If no type is provided, text will be used by default. Each key should be the Android resource name of a target UI element and each value should be the text input for that element. Values are only permitted for text type elements, so no value should be specified for click type elements. For example, use
        --robo-directives text:username_resource=username,text:password_resource=password
    to provide custom login credentials for your app, or
        --robo-directives click:sign_in_button=
    to instruct Robo to click on the sign in button. To learn more about Robo test and robo_directives, see https://firebase.google.com/docs/test-lab/command-line#custom_login_and_text_input_with_robo_test. Caution: You should only use credentials for test accounts that are not associated with real users.

DEPRECATED DEVICE DIMENSIONS FLAGS

    --device-ids=MODEL_ID,[MODEL_ID,...], -d MODEL_ID,[MODEL_ID,...]
        The list of MODEL_IDs to test against (default: one device model determined by the Firebase Test Lab device catalog; see TAGS listed by the $ gcloud firebase test android devices list command).

    --locales=LOCALE,[LOCALE,...], -l LOCALE,[LOCALE,...]
        The list of LOCALEs to test against (default: a single locale determined by the Firebase Test Lab device catalog).

    --orientations=ORIENTATION,[ORIENTATION,...], -o ORIENTATION,[ORIENTATION,...]
        The device orientation(s) to test against (default: portrait). ORIENTATION must be one of: portrait, landscape.

    --os-version-ids=OS_VERSION_ID,[OS_VERSION_ID,...], -v OS_VERSION_ID,[OS_VERSION_ID,...]
        The list of OS_VERSION_IDs to test against (default: a version ID determined by the Firebase Test Lab device catalog).

LIST COMMAND FLAGS

    --filter=EXPRESSION
        Apply a Boolean filter EXPRESSION to each resource item to be listed. If the expression evaluates True then that item is listed. For more details and examples of filter expressions run $ gcloud topic filters. This flag interacts with other flags that are applied in this order: --flatten, --sort-by, --filter, --limit.

    --limit=LIMIT
        The maximum number of resources to list. The default is unlimited. This flag interacts with other flags that are applied in this order: --flatten, --sort-by, --filter, --limit.

    --page-size=PAGE_SIZE
        Some services group resource list output into pages. This flag specifies the maximum number of resources per page. The default is determined by the service if it supports paging, otherwise it is unlimited (no paging). Paging may be applied before or after --filter and --limit depending on the service.

    --sort-by=[FIELD,...]
        A comma-separated list of resource field key names to sort by. The default order is ascending. Prefix a field with ``~'' for descending order on that field. This flag interacts with other flags that are applied in this order: --flatten, --sort-by, --filter, --limit.

    --uri
        Print a list of resource URIs instead of the default output.

GCLOUD WIDE FLAGS
These flags are available to all commands: --account, --configuration, --flatten, --format, --help, --log-http, --project, --quiet, --trace-token, --user-output-enabled, --verbosity. Run $ gcloud help for details.

EXAMPLES
To invoke a robo test lasting 100 seconds against the default device environment, run:
    $ gcloud firebase test android run --app APP_APK --timeout 100s
When specifying devices to test against, the preferred method is to use the --device flag. For example, to invoke a robo test against a virtual, generic MDPI Nexus device in landscape orientation, run:
    $ gcloud firebase test android run --app APP_APK \
        --device model=NexusLowRes,orientation=landscape
To invoke an instrumentation test against a physical Nexus 4 device (MODEL_ID: mako) which is running Android API level 19 in French, run:
    $ gcloud firebase test android run --app APP_APK --test TEST_APK \
        --device model=mako,version=19,locale=fr
To test against multiple devices, specify --device more than once:
    $ gcloud firebase test android run --app APP_APK --test TEST_APK \
        --device model=Nexus4,version=19 \
        --device model=Nexus4,version=21 \
        --device model=NexusLowRes,version=25
You may also use the legacy dimension flags (deprecated) to specify which devices to use. Firebase Test Lab will run tests against every possible combination of the listed device dimensions. Note that some combinations of device models and OS versions may not be valid or available in Test Lab. Any unsupported combinations of dimensions in the test matrix will be skipped. For example, to execute a series of 5-minute robo tests against a very comprehensive matrix of virtual and physical devices, OS versions, locales and orientations, run:
    $ gcloud firebase test android run --app APP_APK --timeout 5m \
        --device-ids=mako,NexusLowRes,Nexus5,g3,zeroflte \
        --os-version-ids=19,21,22,23,24,25 --locales=en_GB,es,fr,ru,zh \
        --orientations=portrait,landscape
The above command will generate a test matrix with a total of 300 test executions, but only the subset of executions with valid dimension combinations will actually run your tests. Controlling Results Storage By default, Firebase Test Lab stores detailed test results for a limited time in a Google Cloud Storage bucket provided for you at no charge. If you wish to use a storage bucket that you control, or if you need to retain detailed test results for a longer period, use the --results-bucket option. See https://firebase.google.com/docs/test-lab/analyzing-results#detailed for more information. Detailed test result files are prefixed by default with a timestamp and a random character string. If you require a predictable path where detailed test results are stored within the results bucket (say, if you have a Continuous Integration system which does custom post-processing of test result artifacts), use the --results-dir option. Note that each test invocation must have a unique storage location, so never reuse the same value for --results-dir between different test runs. Possible strategies could include using a UUID or sequence number for --results-dir. For example, to run a robo test using a specific Google Cloud Storage location to hold the raw test results, run:
    $ gcloud firebase test android run --app APP_APK \
        --results-bucket=gs://my-bucket \
        --results-dir=my/test/results/<unique-value>
To run an instrumentation test and specify a custom name under which the history of your tests will be collected and displayed in the Firebase console, run:
    $ gcloud firebase test android run --app APP_APK --test TEST_APK \
        --results-history-name='Excelsior App Test History'
Argument Files All test arguments for a given test may alternatively be stored in an argument group within a YAML-formatted argument file. The ARG_FILE may contain one or more named argument groups, and argument groups may be combined using the include: attribute (Run $ gcloud topic arg-files for more information). The ARG_FILE can easily be shared with colleagues or placed under source control to ensure consistent test executions. To run a test using arguments loaded from an ARG_FILE named excelsior_args, which contains an argument group named robo-args:, use the following syntax:
    $ gcloud firebase test android run path/to/excelsior_args:robo-args