Implementation of session suspend feature.

plainbox.impl.session.suspend – session suspend support

This module contains classes that can suspend an instance of SessionState. The general idea is that SessionSuspendHelper knows how to describe the session and SessionResumeHelper knows how to recreate the session from that description.

Both of the helper classes are only used by SessionManager and in the the legacy suspend/resume code paths of _LegacySessionState. Applications should use one of those APIs to work with session snapshots.

The design of the on-disk format is not like typical pickle or raw dump of all of the objects. Instead it is designed to create a smart representation of a subset of the data and explicitly support migrations, so that some future version of PlainBox can change the format and still read old sessions (to the extent that it makes sense) or at least reject them with an intelligent message.

One important consideration of the format is that we suspend very often and resume very infrequently so everything is optimized around saving big chunks of data incrementally (all the big job results and their log files) and to keep most of the data we save over and over small.

The key limitation in how the suspend code works is that we cannot really serialize jobs at all. There are two reasons for that, one very obvious and one which is more of a design decision.

The basic reason for why we cannot serialize jobs is that we cannot really, meaningfully serialize the code that runs inside a job. That may the shell command or a call into python module. Without this limitation we would be basically pretending that we are running the same job as before while the job definition has transparently changed and the results would not be sensible anymore.

The design decision is to allow abstract, opaque Providers to offer various types of JobDefinitions (that may be radically different to what current CheckBox jobs look like). This is why the resume interface requires one to provide a full list of job definitions to resume. This is also why the checksum attribute can be implemented differently in non-CheckBox jobs.

As an exception to this rule we _do_ serialize generated jobs. Those are a compromise between ease-of-use of the framework and the external considerations mentioned above. Generated jobs are re-created from whatever results that created them. The framework has special support code for knowing how to resume in light of the fact that some jobs might be generated during the resume process itself.

Serialization format versions

  1. The initial version
  2. Same as ‘1’ but suspends plainbox.impl.session.state.SessionMetaData.app_blob
  3. Same as ‘2’ but suspends plainbox.impl.session.state.SessionMetaData.app_id
  4. Same as ‘3’ but hollow results are not saved and jobs that only have hollow results are not mentioned in the job -> checksum map.
  5. Same as ‘4’ but DiskJobResult is stored with a relative pathname to the log file if session_dir is provided.
  6. Same as ‘5’ plus store the list of mandatory jobs.
plainbox.impl.session.suspend.SessionSuspendHelper

alias of SessionSuspendHelper6

class plainbox.impl.session.suspend.SessionSuspendHelper1[source]

Bases: object

Helper class for computing binary representation of a session.

The helper only creates a bytes object to save. Actual saving should be performed using some other means, preferably using SessionStorage.

This class creates version ‘1’ snapshots.

VERSION = 1
suspend(session, session_dir=None)[source]

Compute suspend representation.

Compute the data that is saved by SessionStorage as a part of SessionStorage.save_checkpoint().

Parameters:
  • session – The SessionState object to represent.
  • session_dir – (optional) The base directory of the session. If this argument is used then it can alter the representation of some objects related to filesystem artefacts. It is recommended to always pass the session directory.
Returns bytes:

the serialized data

class plainbox.impl.session.suspend.SessionSuspendHelper2[source]

Bases: plainbox.impl.session.suspend.SessionSuspendHelper1

Helper class for computing binary representation of a session.

The helper only creates a bytes object to save. Actual saving should be performed using some other means, preferably using SessionStorage.

This class creates version ‘2’ snapshots.

VERSION = 2
suspend(session, session_dir=None)

Compute suspend representation.

Compute the data that is saved by SessionStorage as a part of SessionStorage.save_checkpoint().

Parameters:
  • session – The SessionState object to represent.
  • session_dir – (optional) The base directory of the session. If this argument is used then it can alter the representation of some objects related to filesystem artefacts. It is recommended to always pass the session directory.
Returns bytes:

the serialized data

class plainbox.impl.session.suspend.SessionSuspendHelper3[source]

Bases: plainbox.impl.session.suspend.SessionSuspendHelper2

Helper class for computing binary representation of a session.

The helper only creates a bytes object to save. Actual saving should be performed using some other means, preferably using SessionStorage.

This class creates version ‘3’ snapshots.

VERSION = 3
suspend(session, session_dir=None)

Compute suspend representation.

Compute the data that is saved by SessionStorage as a part of SessionStorage.save_checkpoint().

Parameters:
  • session – The SessionState object to represent.
  • session_dir – (optional) The base directory of the session. If this argument is used then it can alter the representation of some objects related to filesystem artefacts. It is recommended to always pass the session directory.
Returns bytes:

the serialized data

class plainbox.impl.session.suspend.SessionSuspendHelper4[source]

Bases: plainbox.impl.session.suspend.SessionSuspendHelper3

Helper class for computing binary representation of a session.

The helper only creates a bytes object to save. Actual saving should be performed using some other means, preferably using SessionStorage.

This class creates version ‘4’ snapshots.

VERSION = 4
suspend(session, session_dir=None)

Compute suspend representation.

Compute the data that is saved by SessionStorage as a part of SessionStorage.save_checkpoint().

Parameters:
  • session – The SessionState object to represent.
  • session_dir – (optional) The base directory of the session. If this argument is used then it can alter the representation of some objects related to filesystem artefacts. It is recommended to always pass the session directory.
Returns bytes:

the serialized data

class plainbox.impl.session.suspend.SessionSuspendHelper5[source]

Bases: plainbox.impl.session.suspend.SessionSuspendHelper4

Helper class for computing binary representation of a session.

The helper only creates a bytes object to save. Actual saving should be performed using some other means, preferably using SessionStorage.

This class creates version ‘5’ snapshots.

VERSION = 5
suspend(session, session_dir=None)

Compute suspend representation.

Compute the data that is saved by SessionStorage as a part of SessionStorage.save_checkpoint().

Parameters:
  • session – The SessionState object to represent.
  • session_dir – (optional) The base directory of the session. If this argument is used then it can alter the representation of some objects related to filesystem artefacts. It is recommended to always pass the session directory.
Returns bytes:

the serialized data

class plainbox.impl.session.suspend.SessionSuspendHelper6[source]

Bases: plainbox.impl.session.suspend.SessionSuspendHelper5

Helper class for computing binary representation of a session.

The helper only creates a bytes object to save. Actual saving should be performed using some other means, preferably using SessionStorage.

This class creates version ‘6’ snapshots.

VERSION = 6
suspend(session, session_dir=None)

Compute suspend representation.

Compute the data that is saved by SessionStorage as a part of SessionStorage.save_checkpoint().

Parameters:
  • session – The SessionState object to represent.
  • session_dir – (optional) The base directory of the session. If this argument is used then it can alter the representation of some objects related to filesystem artefacts. It is recommended to always pass the session directory.
Returns bytes:

the serialized data

comments powered by Disqus