Monday 20 April 2015

MODELs & Generated Code

Hand cranked codecs are a pain to write and a pain to upgrade, I have written many over the years !  The solution is to generate the code.

An XML model defines the internal model with POJO's that all components can work with eg NewOrderSingle, NewOrderAck, TradeNew. It also defines external models which can be client or exchange, FIX variants or binary protocols such as ETS, Millenium, UTP etc. Finally it defines codecs which specify how to translate external model to/from internal model.

Sample Internal Event for a New Order Single

<Base id="BaseOrderRequest" src="client" extends="CommonClientHeader">
  <Attribute typeId="Instrument"                              name="instrument"   mandatory="Y"     outbound="delegate"/>
  <Attribute typeId="ClientProfile"                           name="client"       mandatory="Y"     outbound="delegate"/>
  <Attribute typeId="viewstring[CLORDID_LENGTH]"    tag="11"  name="clOrdId"      mandatory="Y"     outbound="delegate"/>
  <Attribute typeId="viewstring[CLORDID_LENGTH]"    tag="41"  name="origClOrdId"  mandatory="Y"     outbound="delegate"/>
  <Attribute typeId="viewstring[SECURITYID_LENGTH]" tag="48"  name="securityId"   mandatory="Y"     outbound="delegate"/>
  <Attribute typeId="viewstring[SYMBOL_LENGTH]"     tag="55"  name="symbol"       mandatory="Y"     outbound="delegate"/>
  <Attribute typeId="Currency"                      tag="15"  name="currency"     mandatory="N"     outbound="seperate"/>
  <Attribute typeId="SecurityIDSource"              tag="22"                      mandatory="Y"     outbound="delegate"/>
  <Attribute typeId="UTCTimestamp"                  tag="60"  name="transactTime" mandatory="Y"     outbound="delegate"/>
  <Attribute typeId="UTCTimestamp"                  tag="52"  name="sendingTime"                    outbound="seperate"/>
  <Attribute typeId="Side"                          tag="54"                      mandatory="Y"     outbound="delegate"/>
  <Attribute typeId="viewstring[SRC_LINKID_LENGTH]"           name="srcLinkId"    mandatory="N"     outbound="delegate"/>
</Base>
   
<Base id="OrderRequest" src="client" extends="BaseOrderRequest">
  <Attribute typeId="viewstring[ACCOUNT_LENGTH]"        tag="1"   name="account"          mandatory="N" outbound="delegate"/>
  <Attribute typeId="viewstring[TEXT_LENGTH]"           tag="58"  name="text"             mandatory="N" outbound="delegate"/>
  <Attribute typeId="viewstring[EXDESTINATION_LENGTH]"  tag="100" name="exDest"           mandatory="N" outbound="delegate"/>
  <Attribute typeId="viewstring[SECURITYEXCH_LENGTH]"   tag="207" name="securityExchange" mandatory="N" outbound="delegate"/>
  <Attribute typeId="double"                        tag="44"  name="price"      mandatory="Y"     outbound="seperate"/>
  <Attribute typeId="int"                           tag="38"  name="orderQty"   mandatory="Y"     outbound="seperate"/>
  <Attribute typeId="ExecInst"                      tag="18"                                      outbound="delegate"/>
  <Attribute typeId="HandlInst"                     tag="21"                                      outbound="delegate"/>
  <Attribute typeId="OrderCapacity"                 tag="528"                                     outbound="seperate"/>
  <Attribute typeId="OrdType"                       tag="40"                    mandatory="Y"     outbound="delegate"/>
  <Attribute typeId="SecurityType"                  tag="167"                                     outbound="delegate"/>
  <Attribute typeId="SecurityIDSource"              tag="22"                                      outbound="delegate"/>
  <Attribute typeId="TimeInForce"                   tag="59"                                      outbound="delegate"/>
  <Attribute typeId="BookingType"                   tag="775"                                     outbound="delegate"/>
  <Attribute typeId="long"                                    name="orderReceived" mandatory="Y"  outbound="delegate"/>
  <Attribute typeId="long"                                    name="orderSent"     mandatory="Y"  outbound="delegateGetAndSet"/>
</Base>

<Event id="NewOrderSingle" extends="OrderRequest" src="client">
  <Attribute typeId="viewstring[CLORDID_LENGTH]"    tag="11"  name="clOrdId"     mandatory="Y"     outbound="seperate"/>
</Event>

Sample external definition for a New Order in ETI … note each field must have a dictionary entry which defines its type in the external model.

<Message id="NewOrderRequestSimple"            msgType="10125">
    <Field id="msgSeqNum"               mand="Y"/>
    <Field id="senderSubID"             mand="Y"/>              
   
    <Field id="price"                   mand="Y"/>
    <Field id="senderLocationID"        mand="N"/>
    <Field id="clOrdId"                 mand="Y"/>
    <Field id="orderQty"                mand="Y"/>
    <Field id="filler1c"                len="4"/>   <!-- maxShow tag210 -->
    <Field id="simpleSecurityID"        mand="Y"/>
   
    <Field id="accountType"             mand="N"/>
    <Field id="side"                    mand="Y"/>
    <Field id="priceValidityCheckType"  mand="Y"/>
    <Field id="timeInForce"             mand="Y"/>
    <Field id="execInst"                mand="Y"/>
    <Field id="uniqueClientCode"        mand="N"/>
    <Field id="filler3"                 len="3" comment="pad3"/>     
</Message>

Sample CODEC for a New Order in BSE ETI

<MessageMap id="BaseBSEOrder" messageId="" ignore="true" extends="BaseRequest">
    <Map field="marketSegmentID"><Hook type="encode" code="encodeMarketSegmentID( msg.getInstrument() )"/></Map>

    <Map eventAttr="securityId" field="simpleSecurityID">
        <Hook type="encode" code="encodeSimpleSecurityId( msg.getInstrument() )"/>
        <Hook type="decode" code="_securityId = _builder.decodeUInt()"/>
    </Map>

    <Map field="priceValidityCheckType"><Hook type="encode" code="_builder.encodeByte( (byte)0 )"/></Map>
    <Map field="accountType"><Hook type="encode" code="_builder.encodeByte( (byte)20 )"/></Map>
    <Map field="maxPricePercentage"><Hook type="encode" code="_builder.encodePrice( 0.5 )"/></Map>
    <Map field="senderLocationID"><Hook type="encode" code="_builder.encodeLong( _locationId )"/></Map>
    <Map field="orderCapacity"><Hook type="encode" code="_builder.encodeByte( (byte)1 )"/></Map>
    <Map field="positionEffect"><Hook type="encode" code="_builder.encodeByte( (byte)'C' )"/></Map>
    <Map field="account"><Hook type="encode" code="_builder.encodeStringFixedWidth( _account, 2 )"/></Map>
    <Map field="applSeqIndicator"><Hook type="encode" code="_builder.encodeByte( (byte)0 )"/></Map>
    <Map field="execInst"><Hook type="encode" code="_builder.encodeByte( (byte)2 )"/></Map>
    <Map field="uniqueClientCode">
        <Hook type="encode" code="_builder.encodeStringFixedWidth( _uniqueClientCode, 12 )"/>
        <Hook type="decode" code="_builder.skip( 12 )"/>
    </Map>
</MessageMap>

<MessageMap id="NewLimitOrder"   eventId="NewOrderSingle" messageId="NewOrderRequestSimple" extends="BaseBSEOrder" encodeFunc="encodeNOS">
    <Map field="productComplex"><Hook type="encode" code="_builder.encodeByte( (byte)1 )"/></Map>
    <Hook type="postDecode" code="enrich( msg ) ; msg.setOrdType( OrdType.Limit )"/>
</MessageMap>

Note hooks allow overriding of the default code generation which is based on comparing the internal model dictionary entry with the external model dictionary entry. Map entries are only added for fields that don’t want the default behaviour.

Sample Generated Encoder for NOS :-

public final void encodeNewLimitOrder( final NewOrderSingle msg ) {
    final int now = _tzCalculator.getNowUTC();
    _builder.start( MSG_NewOrderRequestSimple );
    if ( _debug ) {
        _dump.append( "  encodeMap=" ).append( "NewOrderRequestSimple" ).append( "  eventType=" ).append( "NewOrderSingle" ).append( " : " );
    }

    if ( _debug ) _dump.append( "\nField: " ).append( "msgSeqNum" ).append( " : " );
    _builder.encodeUInt( (int)msg.getMsgSeqNum() );
    if ( _debug ) _dump.append( "\nHook : " ).append( "senderSubID" ).append( " : " ).append( "encode" ).append( " : " );
    _builder.encodeUInt( _senderSubID ); // senderSubID;
    if ( _debug ) _dump.append( "\nField: " ).append( "price" ).append( " : " );
    _builder.encodeDecimal( msg.getPrice() );
    if ( _debug ) _dump.append( "\nHook : " ).append( "senderLocationID" ).append( " : " ).append( "encode" ).append( " : " );
    _builder.encodeLong( _locationId );
    if ( _debug ) _dump.append( "\nField: " ).append( "clOrdId" ).append( " : " );
    _builder.encodeStringAsLong( msg.getClOrdId() );
    if ( _debug ) _dump.append( "\nField: " ).append( "orderQty" ).append( " : " );
    _builder.encodeQty( (int)msg.getOrderQty() );
    if ( _debug ) _dump.append( "\nField: " ).append( "filler1c" ).append( " : " );
    _builder.encodeFiller( 4 );
    if ( _debug ) _dump.append( "\nHook : " ).append( "simpleSecurityID" ).append( " : " ).append( "encode" ).append( " : " );
    encodeSimpleSecurityId( msg.getInstrument() );
    if ( _debug ) _dump.append( "\nHook : " ).append( "accountType" ).append( " : " ).append( "encode" ).append( " : " );
    _builder.encodeByte( (byte)20 );
    if ( _debug ) _dump.append( "\nField: " ).append( "side" ).append( " : " );
    _builder.encodeByte( transformSide( msg.getSide() ) );
    if ( _debug ) _dump.append( "\nHook : " ).append( "priceValidityCheckType" ).append( " : " ).append( "encode" ).append( " : " );
    _builder.encodeByte( (byte)0 );
    if ( _debug ) _dump.append( "\nField: " ).append( "timeInForce" ).append( " : " );
    final TimeInForce tTimeInForceBase = msg.getTimeInForce();
    final byte tTimeInForce = ( tTimeInForceBase == null ) ?  DEFAULT_TimeInForce : transformTimeInForce( tTimeInForceBase );
    _builder.encodeByte( tTimeInForce );
    if ( _debug ) _dump.append( "\nHook : " ).append( "execInst" ).append( " : " ).append( "encode" ).append( " : " );
    _builder.encodeByte( (byte)2 );
    if ( _debug ) _dump.append( "\nHook : " ).append( "uniqueClientCode" ).append( " : " ).append( "encode" ).append( " : " );
    _builder.encodeStringFixedWidth( _uniqueClientCode, 12 );
    if ( _debug ) _dump.append( "\nField: " ).append( "filler3" ).append( " : " );
    _builder.encodeFiller( 3 );
    _builder.end();
}


Sample debug log output … essential for debugging the model when testing exchange connectivity.

{  ENCODE msgType=10125  encodeMap=NewOrderRequestSimple  eventType=NewOrderSingle :
Field: msgSeqNum : uint 10,  bytes=4, offset=16, raw=[ 0A 00 00 00 ]
Hook : senderSubID : encode : uint 810701002,  bytes=4, offset=20, raw=[ CA 50 52 30 ]
Field: price : decimal 62.9,  bytes=8, offset=24, raw=[ 80 C8 E9 76 01 00 00 00 ]
Hook : senderLocationID : encode : long 1234567890123456,  bytes=8, offset=32, raw=[ C0 BA 8A 3C D5 62 04 00 ]
Field: clOrdId : stringAsLong 38470008  (len=8),  bytes=8, offset=40, raw=[ 78 01 4B 02 00 00 00 00 ]
Field: orderQty : qty 10,  bytes=4, offset=48, raw=[ 0A 00 00 00 ]
Field: filler1c : filler  len=4,  bytes=4, offset=52, raw=[ 00 00 00 00 ]
Hook : simpleSecurityID : encode : int 1000627,  bytes=4, offset=56, raw=[ B3 44 0F 00 ]
Hook : accountType : encode : byte ^T,  bytes=1, offset=60, raw=[ 14 ]
Field: side : byte ^A,  bytes=1, offset=61, raw=[ 01 ]
Hook : priceValidityCheckType : encode : byte ^@,  bytes=1, offset=62, raw=[ 00 ]
Field: timeInForce : byte ^@,  bytes=1, offset=63, raw=[ 00 ]
Hook : execInst : encode : byte ^B,  bytes=1, offset=64, raw=[ 02 ]
Hook : uniqueClientCode : encode : stringFixedWidth OWN  (len=12),  bytes=12, offset=65, raw=[ 4F 57 4E 00 00 00 00 00 00 00 00 00 ]
Field: filler3 : filler  len=3,  bytes=3, offset=77, raw=[ 00 00 00 ]
} bytes=80

16:15:12.445 [info]
 OUT [exchangeSession1]:
0000 P....'...............PR0...v.......<.b..x.K.......
0050 .......D.......OWN............
0100
     1        10        20        30        40        50
 OUT [exchangeSession1]:
0000 50 00 00 00 8D 27 00 00 00 00 00 00 00 00 00 00 0A 00 00 00 CA 50 52 30 80 C8 E9 76 01 00 00 00 C0 BA 8A 3C D5 62 04 00 78 01 4B 02 00 00 00 00 0A 00
0050 00 00 00 00 00 00 B3 44 0F 00 14 01 00 00 02 4F 57 4E 00 00 00 00 00 00 00 00 00 00 00 00
0100
     1  2  3  4  5  6  7  8  9  10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50
byteCount=80

16:15:12.445 [info]   OUT [exchangeSession1]: NewOrderSingleImpl , clOrdId=38470008, account=, text=, exDest=, securityExchange=, price=62.9, orderQty=10, execInst=null, handlInst=null, orderCapacity=null, ordType=Limit, securityType=, securityIDSource=, timeInForce=Day, bookingType=null, orderReceived=[null], orderSent=[null], instrument=1000627, client=, origClOrdId=, securityId=, symbol=, currency=, transactTime=[null], sendingTime=[null], side=Buy, srcLinkId=, onBehalfOfId=, msgSeqNum=10, possDupFlag=N



Because the code is generated for both encoding and decoding, then the exchange simulator can simulate any exchange in the model.

I wrote the code generator from scratch in a couple of weeks, its completely custom and not general purpose.

The resulting internal POJO's interfaces, and CODEC's for external to internal translation result in over 100,000 lines of generated high quality code.



No comments:

Post a Comment