Old Architecture Notes

Warning

This section needs maintenance

Application Skeleton

This skeleton represents a typical application based on Plainbox. It enumerates the essential parts of the APIs from the point of view of an application developer.

  1. Instantiate plainbox.impl.checkbox.Checkbox then call plainbox.impl.checkbox.Checkbox.get_builtin_jobs() to discover all known jobs. In the future this might be replaced by a step that obtains jobs from a named provider.
  1. Instantiate plainbox.impl.runner.JobRunner so that we can run jobs

  2. Instantiate plainbox.impl.session.SessionState so that we can keep track of application state.

    • Potentially restore an earlier, interrupted, testing session by calling plainbox.impl.session.SessionState.restore()
    • Potentially remove an earlier, interrupted, testing session by calling plainbox.impl.session.SessionState.discard()
    • Potentially start a new test session by calling plainbox.impl.session.SessionState.open()
  3. Allow the user to select jobs that should be executed and update session state by calling plainbox.impl.session.SessionState.update_desired_job_list()

  4. For each job in plainbox.impl.SessionState.run_list:

    1. Check if we want to run the job (if we have a result for it from previous runs) or if we must run it (for jobs that cannot be persisted across suspend)

    2. Check if the job can be started by looking at plainbox.impl.session.JobState.can_start()

      • optionally query for additional data on why a job cannot be started and present that to the user.
      • optionally abort the sequence and go to step 5 or the outer loop.
    3. Call plainbox.impl.runner.JobRunner.run_job() with the current job and store the result.

      • optionally ask the user to perform some manipulation
      • optionally ask the user to qualify the outcome
      • optionally ask the user for additional comments
    4. Call plainbox.impl.session.SessionState.update_job_result() to update readiness of jobs that depend on the outcome or output of current job.

    5. Call plainbox.impl.session.SessionState.checkpoint() to ensure that testing can resume after system crash or shutdown.

  5. Instantiate the selected state exporter, for example plainbox.impl.exporters.json.JSONSessionStateExporter so that we can use it to save test results.

    • optionally pass configuration options to customize the subset and the presentation of the session state
  6. Call plainbox.impl.exporters.SessionStateExporterBase.get_session_data_subset() followed by plainbox.impl.exporters.SessionStateExporterBase.dump() to save results to a file.

  7. Call plainbox.impl.session.SessionState.close() to remove any nonvolatile temporary storage that was needed for the session.

Essential classes

SessionState

Class representing all state needed during a single program session.

Usage

The general idea is that you feed the session with a list of known jobs and a subset of jobs that you want to run and in return get an ordered list of jobs to run.

It is expected that the user will select / deselect and run jobs. This class can react to both actions by recomputing the dependency graph and updating the read states accordingly.

As the user runs subsequent jobs the results of those jobs are exposed to the session with update_job_result(). This can cause subsequent jobs to become available (not inhibited by anything). Note that there is no notification of changes at this time.

The session does almost nothing by itself, it learns about everything by observing job results coming from the job runner (plainbox.impl.runner.JobRunner) that applications need to instantiate.

Suspend and resume

The session can save check-point data after each job is executed. This allows the system to survive and continue after a catastrophic failure (broken suspend, power failure) or continue across tests that require the machine to reboot.

Implementation notes

Internally it ties into plainbox.impl.depmgr.DependencySolver for resolving dependencies. The way the session objects are used allows them to return various problems back to the UI level - those are all the error classes from plainbox.impl.depmgr:

Normally none of those errors should ever happen, they are only provided so that we don’t choke when a problem really happens. Everything is checked and verified early before starting a job so typical unit and integration testing should capture broken job definitions (for example, with cyclic dependencies) being added to the repository.

Implementation issues

  • There is too much checkbox-specific knowledge which really belongs elsewhere. We are working to remove that so that non-checkbox jobs can be introduced later. There is a branch in progress that entirely removes that and moves it to a new concept called SessionController. In that design the session delegates understanding of results to a per-job session controller and exposes some APIs to alter the state that was previously internal (most notably a way to add new jobs and resources).

JobDefinition

Checkbox has a concept of a job. Jobs are named units of testing work that can be executed. Typical jobs range from automated CPU power management checks, BIOS tests, semi-automated peripherals testing to all manual validation by following a script (intended for humans).

Jobs are distributed in plain text files, formated as a loose RFC822 documents where typically a single text file contains a few dozen different jobs that belong to one topic, for example, all bluetooth tests.

Tests have a number of properties that will not be discussed in detail here, they are all documented in plainbox.impl.job.JobDefinition. From the architecture point of view the four essential properties of a job are name, plugin and requires and depends. Those are discussed in detail below.

JobDefinition.name

The name field must be unique and is referred to by other parts of the system (such as whitelists). Typically jobs follow a simple naming pattern ‘category/detail’, eg, ‘networking/modem_connection’. The name must be _unique_ and this is enforced by the core.

JobDefinition.plugin

The plugin field is an archaism from Checkbox and a misnomer (as Plainbox does not have any plugins). In the Checkbox architecture it would instruct the core which plugin should process that job. In Plainbox it is a way to encode what type of a job is being processed. There is a finite set of types that are documented below.

plugin == “shell”

This value is used for fully automated jobs. Everything the job needs to do is automated (preparation, execution, verification) and fully handled by the command that is associated with a job.

plugin == “manual”

This value is used for fully manual jobs. It has no special handling in the core apart from requiring a human-provided outcome (pass/fail classification)

plugin == “resource”

This value is used for special “data” or “environment” jobs. Their output is parsed as a list of RFC822 records and is kept by the core during a testing session.

They are primarily used to determine if a given job can be started. For example, a particular bluetooth test may use the _requires_ field to indicate that it depends (via a resource dependency) on a job that enumerates devices and that one of those devices must be a bluetooth device.

plugin == “user-interact”

For all intents and purposes it is equivalent to “manual”. The actual difference is that a user is expected to perform some physical manipulation before an automated test.

plugin == “user-verify”

For all intents and purposes it is equivalent to “manual”. The actual difference is that a user is expected to perform manual verification after an automated test.

JobDefinition.depends

The depends field is used to express dependencies between two jobs. If job A has depends on job B then A cannot start if B is not both finished and successful. Plainbox understands this dependency and can automatically sort and execute jobs in proper order. In many places of the code this is referred to as a “direct dependency” (in contrast to “resource dependency”)

The actual syntax is not strictly specified, Plainbox interprets this field as a list of tokens delimited by comma or any whitespace (including newlines).

A job may depend on any number of other jobs. There are a number of failure modes associated with this feature, all of which are detected and handled by Plainbox. Typically they only arise when during Checkbox job development (editing actual job files) and are always a sign of a human error. No released version of Checkbox or Plainbox should ever encounter any of those issues.

The actual problems are:

  • dependency cycles, where job either directly or indirectly depends on itself
  • missing dependencies where some job refers to a job that is not defined anywhere.
  • duplicate jobs where two jobs with the same name (but different definition) are being introduced to the system.

In all of those cases the core removes the offending job and tries to work regardless of the problem. This is intended more as a development aid rather than a reliability feature as no released versions of either project should cause this problem.

JobDefinition.command

The command field is used when the job needs to call an external command. Typically all shell jobs define a command to run.

“Manual” jobs can also define a command to run as part of the test procedure.

JobDefinition.user

The user field is used when the job requires to run as a specific user (e.g. root).

The job command will be run via pkexec to get the necessary permissions.

JobDefinition.environ

The environ field is used to pass additional environmental keys from the user session to the new environment set up when the job command is run by another user (root, most of the time).

The actual syntax is not strictly specified, Plainbox interprets this field as a list of tokens delimited by comma or any whitespace (including newlines).

comments powered by Disqus