rk3588-game
/
android13
3
0
Fork 0
Code Issues Pull Requests Packages Projects Releases Wiki Activity
android13/external/parameter-framework/upstream/test/functional-tests/plan.md

14 KiB
Raw Permalink Blame History

Parameter Framework (PF) test plan

Notes

  • The unit test code MAY be able to list implemented scenarios in this same BDD format in order to check scenario coverage.
  • All scenarios implicitly start by "GIVEN a parameter framework".
  • If a scenario does not specify a config files, the following minimal files SHOULD be used:

Top level file (replace {structurePath} by its value):

<?xml version='1.0' encoding='UTF-8'?>
<ParameterFrameworkConfiguration SystemClassName='test'
                                 TuningAllowed='false' ServerPort='1'>
    <SubsystemPlugins/>
    <StructureDescriptionFileLocation Path='{structurePath}'/>
</ParameterFrameworkConfiguration>

Structure file:

<?xml version='1.0' encoding='UTF-8'?>
<SystemClass Name='test'>
    <Subsystem Name='test' Type='Virtual'>
        <ComponentLibrary/>
        <InstanceDefinition>
            <BooleanParameter Name="placeholder"/>
        </InstanceDefinition>
    </Subsystem>
</SystemClass>

Starting the Parameter Framework

  • set a logger

    • Scenario: Default logger

      • WHEN no logger is set
      • THEN start should succeed

    • Scenario: No logger

      • WHEN a nullptr logger is set
      • THEN start should succeed

    • Scenario: Logger should receive info and warnings

      • GIVEN config files that emit warnings
      • GIVEN a logger that store logs
      • WHEN the record logger is set
      • THEN start should succeed
      • AND_THEN the logger should have stored info and warning log

    • Scenario: Unset logger

      • GIVEN config files that emit warnings
      • GIVEN a logger that store logs
      • WHEN the record logger is set
      • AND_WHEN a nullptr logger is set
      • THEN start should succeed
      • AND_THEN the record logger should NOT have stored any info or warning log

  • set/get various properties before startup:

    • "force no remote interface"

      • Scenario: Tuning OK

        • GIVEN config files with tuning enabled on a valid port
        • WHEN the remote interface is NOT forbiden
        • THEN start should succeed

      • Scenario: Invalid tunning

        • GIVEN config files with tuning enabled on an invalid port (0?)
        • WHEN the remote interface is NOT forbiden
        • THEN start should fail

      • Scenario: Forbiden invalid tunning

        • GIVEN config files with tuning enabled on an invalid port (0?)
        • WHEN the remote interface is forbiden
        • THEN start should succeed

    • "failure on missing subsystem"

      • Scenario: Plugin OK

        • GIVEN config files with a valid plugin
        • WHEN the missing subsystem policy is set to failure
        • THEN start should succeed

      • Scenario: Wrong plugin

        • GIVEN config files with a invalid plugin
        • WHEN the missing subsystem policy is set to failure
        • THEN start should fail

      • Scenario: Wrong plugin but fallback

        • GIVEN config files with a invalid plugin
        • WHEN the missing subsystem policy is set to success
        • THEN start should succeed

    • "failure on failed settings load"

      • Scenario: Settings OK

        • GIVEN config files with a valid settings file
        • WHEN settings load failure is requested to abort start
        • THEN start should succeed

      • Scenario: Wrong settings

        • GIVEN config files with a invalid (non existant?) settings file
        • WHEN settings load failure is requested to abort start
        • THEN start should fail

      • Scenario: Wrong settings but ignore

        • GIVEN config files with a invalid (non existant?) settings file
        • WHEN settings load failure is requested to abort start
        • THEN start should succeed

    • "schema folder location" and "validate schema on start"

      • Scenario: Schema OK

        • GIVEN config files with correct default schema location
        • WHEN schema folder location is left to default
        • WHEN schema validation is enabled
        • THEN start should succeed

      • Scenario: Inexisting schemas

        • GIVEN config files with correct default schema location
        • WHEN schema folder location is set to an invalid location (/doesNotExist ?)
        • WHEN schema validation is enabled
        • THEN start should fail

      • Scenario: Inexisting ignored schemas

        • GIVEN config files with correct default schema location
        • WHEN schema folder location is set to an invalid location (/doesNotExist ?)
        • WHEN schema validation is disabled
        • THEN start should succeed

      • Scenario: Inexisting used schemas

        • GIVEN config files with custom schema location (./shemaCustomFolderName)
        • WHEN schema folder location is set to the custom shema location
        • WHEN schema validation is enabled
        • THEN start should succeed

      • Scenario: Schema OK, config KO

        • GIVEN incorect config files with correct default schema location
        • WHEN schema validation is enabled
        • THEN start should fail

Error cases/special cases

  • invalid configuration or a file can't be read:

    • Scenario: invalid top level config file

      • GIVEN an incorect (empty ? wrong balisa ?) config top level file.
      • THEN start should fail

    • Scenario: invalid structure

      • GIVEN an incorect (empty ? wrong balisa ?) structure file.
      • THEN start should fail

    • Scenario: invalid settings

      • GIVEN an incorect (empty ? wrong balisa ?) settings file.
      • THEN start should fail

  • plugins can't be found (unless silenced by the user)

    • Done in "failure on missing subsystem"

  • usage of <xi:include .../>

    • Scenario: Arbitrary xincluded node
      • GIVEN a top level file xincluding a file with the "SettingsConfiguration" balisa
      • THEN start should succeed

  • plugins:

    • non-empty "Location" attribute
      • Scenario: Plugin location OK
        • GIVEN a top level file with a plugin folder location attribute
        • THEN start should succeed
    • empty "Location" attribute

      • relative path

        • Scenario: Empty Location, standard plugin install

          • GIVEN config files with an empty location attribute
          • GIVEN a plugin installed in a dlopen standard path
          • GIVEN that "Location/Plugin[name]" is the plugin name
          • THEN start should succeed

        • Scenario: Empty Location, non standard plugin install

          • GIVEN config files with an empty location attribute
          • GIVEN a plugin NOT installed in a dlopen standard path
          • GIVEN that "Location/Plugin[name]" is the plugin name
          • THEN start should fail

      • absolute path

        • Scenario: Empty Location, plugin absolute path

          • GIVEN config files with an empty location attribute
          • GIVEN a plugin
          • GIVEN that "Location/Plugin[name]" is the plugin absolute path
          • THEN start should succeed

        • Scenario: Empty Location, plugin non existing path

          • GIVEN config files with an empty location attribute
          • GIVEN that "Location/Plugin[name]" is an non existing path (/libDoesNotExist.so?)
          • THEN start should fail

Logging

  • info and warning
    • Partly covered by the "set a logger" section
    • Scenario: Plugin info and warning log
      • GIVEN a plugin that log a unique string as info and warning
      • GIVEN a logger that store logs
      • WHEN the record logger is set
      • THEN start should succeed
      • AND_THEN the logger should have stored the plugin specific info and warning log

Basic Domains & Configurations management

  • Scenario: create and delete an empty domain

    • GIVEN a started parameter framework without domains
    • THEN creating a domain with an arbitrary name ("domain1"?) should succeed
    • (AND_THEN the domain list should contain the domain name)
    • THEN deleting the domain should succeed

  • Scenario: create and delete a configuration

    • GIVEN a started parameter framework with a domain
    • THEN creating a configuration in this domain with an arbitrary name ("config1") should succeed
    • (AND_THEN the domain configuration list should contain the configuration name)
    • THEN deleting the configuration should succeed
    • (AND_THEN the domain configuration list should not contain the configuration name)

  • Scenario: delete a domain with configurations

    • GIVEN a started parameter framework with a domain and several configuration (3?)
    • THEN deleting the domain should succeed
    • (AND_THEN the domain list should not contain the domain name)

  • Scenario: delete all domains

    • GIVEN a started parameter framework with several domains
    • THEN deleting all domains should succeed
    • (AND_THEN the domain list should be empty)

  • Scenario: rename a domain

    • GIVEN a started parameter framework with a domain and several configuration (3?)
    • THEN renaming the domain to an arbitrary name should succeed

Error cases/special cases

  • Scenario: create a domain with an already-existing name

    • GIVEN a started parameter framework with a domain (arbitrary name)
    • THEN creating a second domain with the same name than the first should fail

  • Scenario: create a configuration with an already-existing name

    • GIVEN a started parameter framework with a domain containing one configuration (arbitrary name)
    • THEN creating a second configuration in this domain with the same name than the fist should fail

  • delete a non-existent domain

    • GIVEN a started parameter framework without domain
    • THEN deleting an arbitrary named domain should fail

  • delete a non-existent configuration

    • GIVEN a started parameter framework one domain
    • THEN deleting in this domain a arbitrary named configuration should fail

  • list configuration of a unknown domain

    • GIVEN a started parameter framework without domain
    • THEN listing configuration of an arbitrary named domain should fail

  • Scenario: rename a domain to an already used name

    • GIVEN a started parameter framework with two domains with arbitrary different names ("domain1" and "domain2")
    • THEN renaming first domain ("domain1") to the name of the second one ("domain2") should fail

  • Scenario: rename a configuration to an already used name

    • GIVEN a started parameter framework with one domain containing two configuration with arbitrary different names ("config1" and "config2")
    • THEN renaming first configuration ("config1") to the name of the second one ("config2") should fail

  • Scenario: rename a domain to its own name

    • GIVEN a started parameter framework one arbitrary named domain
    • THEN renaming the domain to its own name should succeed

  • rename a configuration to its own name

    • GIVEN a started parameter framework a domain and one configuration
    • THEN renaming the configuration to its own name should succeed

Domain modification

  • set/get sequence awareness
  • add/list/remove elements to a domain
  • (split domain ?)
  • (listBelongingDomains ?)
  • (listAssociatedDomains ?)
  • (listAssociatedElements ?)
  • (listConflictingElements ?)
  • (listRogueElements ?)

Error cases/special cases

  • add (or remove) a non-existent element
  • add (or remove) a parameter to (or from) a non-existent domain

Configurations modification

  • set a configuration rule; get it back
    • trivial rules, complex rules (fuzzing ?)
    • clear a rule
  • save/restore a configuration
  • set/get element sequence
  • set/get a parameter for a given configuration
  • set a parameter belonging to a domain and save a configuration

Error cases

  • set a wrongly-formatted rule
  • set a rule using an unknown criterion (or unknown criterion value)
  • set/get a rule to a non-existent domain (or configuration)

Criteria

  • create a criterion
    • exclusive
    • inclusive
  • (list criteria)
  • set a criterion; get a criterion value
    • exclusive criterion
    • inclusive criterion
      • special case: "none" value

Error cases/special cases

  • duplicated criterion name (with the same type/with a different type)
  • duplicated criterion value
  • max supported number of criterion values

Settings Import/Export

  • import xml domains
    • w/ or w/o settings
    • from a file/from a string
    • all/single
  • export xml domains
    • w/ or w/o settings
    • to a file/from a string
    • all/single

Error case/special cases

  • invalid settings
    • invalid xml (should we use a fuzzing framework ?)
    • valid xml but not matching the XSD (ditto)
    • valid but unusable data:
      • unknown criterion/criterion values
      • unknown parameters
  • unreadable/unwritable file
  • single domain import overwrite
    • fail if overwrite not requested
    • succeed if overwrite requested

Parameters

  • structures are correctly instantiated
    • parameters in the "instance definition"
    • component types instantiation
    • parameter blocks
    • bit blocks
    • arrays
  • check parameter mappings are correctly applied
    • at all levels possible (subsystem, component type, component instance, block, parameter...)
    • context mappings, amends, instantiation mappings
  • set/get a parameter
  • parameter handles
  • test on parameter values
    • booleans
    • bit blocks
    • numeric types ranges
    • fixed-point tests
    • string length

Configuration application

  • apply configurations
    • sequence aware domains
  • forced sync

Misc/to be classified

  • tuning mode
  • auto sync
  • remote interface instantiation
  • value space (raw/real)/format (dec/hex)

Error cases