Managing Identification

Section - TM Language Reference, the asterisk(*) operator

Transformation Manager has a number of built-in facilities which control the finding or creation of objects in the target. These include modifiers on individual attributes (principally *, ! - and +) together with the ability to manage the identification information which is present in the data model itself. 

In general identification is relatively simple - here is an example. In the transform there are identifying attributes on the target side - customer number identifies 'Account'.

 

TRANSFORM Account <- Account ;
    *customerNum := customerNum ;
    balance      := 100 + balance ;
END_TRANSFORM;

 

Thus at run-time this will find or create 'Account' elements using customerNum as the identifying key and will then set the balance from the source side, having added 100.

However various different systems impose different limitations on the kind of identification of which can be meaningfully carried out.

 

Non-hierarchical systems

Non-hierarchical systems include JDBC and our built-in Java been adaptor, together with non-hierarchical generic systems - for example a target adaptor writing to CVS (comma, separated values). In these systems it usually possible to create 'Account' items entirely in accordance with the identification information - the only limitations tend to occur in JDBC. Here each instance of it an 'Account' is represented by a row in a database. Where the table has primary key or foreign key constraints which has been ignored in the transform - either because the transform author has used the '-' symbol to cancel built in a identification, or because the meta data loaded into Transformation Manager representing the model is out of step with the real target (and thus does not include all the constraints which are present in the database itself) then insertion or update of the row may be impossible. 

 

Hierarchical systems

Hierarchical systems are more complex. Here the actual hierarchical structure of the data - which of course in XML is actually visible in a text editor - may conflict with or modify the behaviour of the identification. 

The basic problem is - 'to what extent is the identification within context'?. Here is a simple example - suppose the target has already built this structure:

<Bank>
<Branch ref='456712'>
<Account customerNum = '125678' balance ='233'/>
<Account customerNum = '445678' balance ='133'/>
</Branch>
<Branch ref='456713'>
<Account customerNum = '676768' balance ='233'/>
<Account customerNum = '565566' balance ='133'/>
</Branch>
<Bank>
 

but the next customerNum to arrive is 676768.

There are now two reasonable ways we may want to interpret the identification :

  1. We could assume that customerNum is a universal identifier. If this is the case then we will want to update the Account node which is included in the second Branch.
  2. We could assume that customerNum only identifiers within the scope of each particular branch. In this case we may be wanting to add a new customer to the first branch.

As you would expect SML allows both interpretations, both have their validity and application.

The way this is resolved in practice is as follows:

  1. In the special case that the TRANSFORM Account <- Account is called as a result of a cascade operation to its natural XML parent (i.e. in this case Branch) then the identification is assumed to be in the context of its parent. In other words there may well be more than one Account in the resulting file with the same customer number.
  2. Otherwise SML assumes that identification is universal - i.e. that there should only be one Account with each customer number.

Thus to get the first affect (local identification) you should write something like:

TRANSFORM Account <- Account [dependent]
    *customerNum := customerNum ;
    balance      := 100 + balance ;
END_TRANSFORM;

TRANSFORM Branch <- Branch
    Account      := Account ;
END_TRANSFORM;
 

To get a universal identification you should avoid this type of cascade call and either write:

TRANSFORM Account <- Account
    *customerNum := customerNum ;
    balance      := 100 + balance ;
END_TRANSFORM;
 

as an independent transform its own right, or ensure that the Branch branch transform also creates the Account objects:

TRANSFORM Branch <- Branch
    FOR_EACH Individual_Account over Account; 
    BEGIN
        Account[idx].*customerNum := Individual_Account.].*customerNum;
        Account[idx].balance      := 100 + Individual_Account.].balance;
    END;
END_TRANSFORM;
 

Finally note that if you want the first type behaviour identification within context) in the second type of map with an iterator then you can use the special function GETTARGETREF to indicate that Account is identified within the context of the Branch as follows:

TRANSFORM Branch <- Branch
    FOR_EACH Individual_Account over Account; 
    BEGIN
        Account[idx].*customerNum := Individual_Account.].*customerNum;
        Account[idx].*iBranch     := GETTARGETREF();
        Account[idx].balance      := 100 + Individual_Account.].balance;
    END;
END_TRANSFORM;
 

Further notes:

The function ISSEENBEFORE is often, especially for XML targets, is very useful supplement or replacement for identification. This ISSEENBEFORE function enables the transform to act differently for each new item.

The special attribute $ID can be used to provide identification, and hence control over the number of items created, for any node which in reality has no identifying attributes.

For example:

*$ID := CustomerNum;

The $ID attribute will never appear in the final output. 

Mixing identification and nine identification within the same transform is not a supported design pattern. In other words this transform is not going to generate good code:

TRANSFORM Account <- Account;
    IF ( some condition) THEN
        *customerNum := *customerNum;
    ELSE
        customerNum  := newNumberFunction();
    END_IF;
        balance      := 100 + balance ;
END_TRANSFORM;
 

If you wish to write transforms to achieve this effect then you should use the 'mode' functionality to divide a transform into two components. Add one transform to be run when identification is relevant - add another when the identification is not relevant. In this example the code will probably look something like:

TRANSFORM Account <- Account [ dependent] mode="Id"
    *customerNum := customerNum ;
    balance      := 100 + balance ;
END_TRANSFORM;

TRANSFORM Account <- Account [ dependent] mode="New"
    customerNum  := newNumberFunction();
    balance      := 100 + balance ;
END_TRANSFORM;

TRANSFORM Branch <- Branch
    IF (some condition) THEN
        Account  := Account + PushMode('ID');
    ELSE
        Account  := Account + PushMode('New');
    END;
END_TRANSFORM;
 

If the condition can be evaluated for each branch.

If the condition is to be evaluated for each account then the pattern in the Branch transform will look similar to:

TRANSFORM Branch <- Branch
    FOR_EACH Individual_Account over Account; 
    BEGIN
    IF (Individual_Account. some condition) THEN
        Account := Individual_Account + PushMode('ID');
    ELSE
        Account := Individual_Account + PushMode('New');
    END;
END_TRANSFORM;