TMTransformExecutor is a long lived component. It is possible to open and close adapters owned by a TMTransformExecutor during its lifetime. In addition the lifetime of each open adapter may be divided into sets of what we call 'sessions'. These divide its lifetime into a series of open sessions during which "blocks" of data are being transformed. There is no limit to the depth of this sub-division.
The precise activity performed by each user adapter as each session (of any particular level) begins and ends is entirely user defined. Typical 'beginSession' actions could include 'connecting' to a new data source, or beginning some kind of transaction, (or doing nothing at all). Typical 'endSession' activities might be 'disconnecting' or ending a transaction. The only requirement is that sessions should be properly nested - i.e. that each 'beginSession' is followed by an 'endSession' - to end normally or another 'beginSession'. Calling a second 'beginSession' at the same level as an already open Session is used to force the creation of a new session and the loss of the previous session at that level. (Thus there is no need for an abort Session operation.) However these semantics cannot be enforced by Transformation Manager except for its own concept of 'partitions' and 'divisions' (see below). Note finally that TM's session control is not restrictive - generic adapter's may implement any of these operations outside the interfaces required by Transformation Manager.
Once these user operations have been defined all TMTransformExecutor's implementation of 'beginSession' and 'endSession' does is to call through to 'beginSession' and 'endSession' on each of its current open adapters in turn. The adapters are ordered is an order of adapter opening.
While 'endSession' takes no parameters, other than the level of session it is ending, 'beginSession' also takes a java.util.Map. This map is keyed into whatever string data required to "begin a session" at that level again this is user defined. Where a generic adapter has been implemented to co-operate with Transformation Managers GUI interfaces then it is possible, using the call 'getBeginSessionNames' to discover the keys required for any particular adapter - see the interface TMGenericInstance for further details.
Finally note that 'beginSession' returns a boolean. This should be false if no action is taken, true if any action is taken. This allows the Transformer to itself finally return a result for its 'beginSession' operation - which will only be 'true' if any action has been taken by any adapter.
"Partitions" and "Divisions"
Transformation Manager itself may be set to specially respond itself to two levels of session. These we have termed "partition" and "division". These are not hard coded to any particular level of session. For flexibility TMTransformExecutor contains methods to set and get the levels at which 'beginSession' and 'endSession' are to be interpreted internally as instructions to begin or end 'partitions' or 'divisions'. For example, the call setRecognisedDivisionLevel(3) tells TMTransformExecutor to interpret beginSession(3) and endSession(3) as the beginning and ending a TMTransformExecutor "division."
Note that 'beginSession' and 'endSession' events are still also called on all open adapters even if they are interpreted by TMTransformExecutor. In the case of 'beginSession' place any special processing within TMTransformExecutor after 'beginSession' has been called on the other adapters. In the case of 'endSession' TMTransformExecutor's own processing take place before that in the adapters.
Transformation Architecture with Partitions and Division
In order to understand what 'divisions' and 'partitions' actually do it is necessary to have some understanding of the internal architecture of the transformation engine - at least when it is using generic adapters and configured in long-life mode. When internal adapters are in use other caching patterns and optimisations take place.
In long-life mode every call to TMTransformExecutor's transform method produces either zero, one or a number of data handles. Before returning to control to the user TMTransformExecutor will attempt to "flush" these handles out to the user's generic implementation by calling the finalize method. However for reasons of efficiency it will only flush out non-identified handles - handles which have identification are retained internally. The lifetime of this internal retention is the lifetime of a partition. Thus ending a partition will have the effect of flushing identifying handles through the user layer. Thus by ending, or by not ending, partitions the user may control how often identifying handles are flushed out to the user's implementation of the generic adapter. Note that all handles are always flushed out when an adapter closes - thus 'partitions' are optional.
The concept of the 'division', however it is quite different. One of the advanced features of Transformation Manager is its treatment of batched data. Transformation Manager understands that batched data may not necessary arrive in the right order, or may make references to other pieces of data in other batches. To do this it holds light weight references between batches. Transformation Manager chooses to retain this data based on its analysis of value-based relationships.
The lifetime of this lightweight data is known as a "division". Ending a division thus removes from memory any lightweight references that may be used between batches. Note that divisions are only meaningful if TMTransformExecutor::setSupportsCrossSessionHandles has been called. This call simply turns on transformation Manager's ability to remember references. Otherwise such references are never remembered.
Stalled Handles
When Transformation Manager is holding forward references some of its data handles may get placed into a state known as "stalled." These stalled handles cannot be written out through the user layer because they contain unresolved mandatory references. In relational database terms these are best thought of as lacking data for mandatory foreign key columns. Such mandatory references may, of course, arrive in later transforms - if they do then the stalled handle status is cleared and the handle itself automatically written out as appropriate. However stalled handles may persist until the adapter closes at which point their continued existence may prove a problem. Thus three methods exist to help handle the pool of stalled handles.
The method getStalledHandleDescriptions returns descriptions of those handles that are stalled - including the domains and reason for their stalled status. The method TMTransformExecutor:destroyStalledHandles may be used to remove these from memory at any time - the data in the handles is simply discarded. The method clearStalledHandleFlags may be used to remove the stalled status of these handles - the next call to end a partition will then attempt to flush them through to the user layer (this may result in faults at the user layer level as it attempts to deal with unresolved mandatory references).
If an adapter is closed and stalled handles are present then all normal handles are first flushed through to the user layer - then an exception will be thrown to describe what is stalled.
In-Built Adapters and Sessions
Note that today the in-built adapters for databases Java and XML takes no special action on the beginning or ending of sessions at any level. However this will change.