Java API Extras

This document is a collection of articles that compliment the Algorithm Developer's Guide.

Reading Ember Journal

Here is a code snippet that reads the Ember Journal and prints all trade the messages found inside:

try (EmberBinaryJournalReader reader = EmberAppHelper.createObjectJournalReader(config)) {  
    while (reader.read(this::process))  
        ;  
}

...

void process(Object message) {  
    if (message instanceof OrderTradeReportedEvent)
       System.out.println(message);
}

We use the EmberBinaryJournalReader class to read the Journal. The process method checks if the message is an instance of OrderTradeReportedEvent and prints the message if it is.

Complex Algorithm Configuration

Ember configuration uses the HOCON format through the Typesafe config library for Java. This library supports reading POJO (Plain Old Java Object) Java objects from HOCON, including nested objects and lists of objects. In certain cases, you may want to store Map-like settings without corresponding POJO classes. This section shows how to achieve this.

Suppose we want to configure a couple of maps where each key has an associated list of values:

1. Create a PartiesConfig class containing two Map attributes: producers and consumers, where each key has an associated list of values.

   public class PartiesConfig {  
       private Map <String, List<String>> producers;  
       private Map <String, List<String>> consumers;
   }

2. In the HOCON configuration file, provide the necessary settings for PartiesConfig. Use the following format to represent the key-value pairs for producers and consumers:

   parties {  
   producers {  
       GOLD: [ PRODUCER1, PRODUCER2 ]  
       SILVER: [ PRODUCER3, PRODUCER4, PRODUCER5 ]  
   }  
   consumers {  
       TIER_1: [ CONSUMER1, CONSUMER2]  
       TIER_2: [ CONSUMER3 ]  
   }  
   }

3. Create the MyAlgorithmFactory class, which serves as an example of an algorithm factory that uses PartiesConfig.

   public class MyAlgorithmFactory extends AbstractAlgorithmFactory {
       @Required  
       private PartiesConfig parties;
   
       public void setParties(PartiesConfig parties) {  
           this.parties = parties;  
       }
       
       @Override  
       public MyAlgorithm create(AlgorithmContext context) {  
           return new MyAlgorithm(context,  parties));
       }   
   }

4. To handle the lack of direct support for Maps in the TypeSafe Config library, let PartiesConfig implement the ConfigAware interface. This will enable the class to access the library API directly and build the required maps.

   public class PartiesConfig implements ConfigAware {  
   
       @Required  
       private Map <String, List<String>> producers;  
   
       @Required  
       private Map <String, List<String>> consumers;  
   
   
       public Map<String, List<String>> getProducers() {  
           return producers;  
       }  
   
       public Map<String, List<String>> getConsumers() {  
           return consumers;  
       }  
       
       @Override // ConfigAware interface
       public void init(Config config) {  
           producers = loadMap(config.getConfig("producers"));  
           consumers = loadMap(config.getConfig("consumers"));
       }
   
       private static Map<String,List<String>> loadMap (Config config) {  
           Map<String,List<String>> result = new HashMap<>();  
           for (String key : config.root().keySet()) {  
               result.put(key, config.getStringList(key));
           }  
           return result;
       }  
   }

Order ID Generation

Ember uses a compose order identity, where each order is uniquely identified by a pair of {OrderSourceID, OrderID}. When algorithms submit orders into the Order Management System (OMS), the OrderSourceID corresponds to the algorithm deployment ID (e.g., "TWAP"). This section describes the OrderID generation method used in algorithms.

API Helper Method: makeMarketOrder()

The class AbstractAlgorithm provides several helper methods for submitting new orders, one of which is makeMarketOrder(). This method simplifies the process of creating a new market order with specified parameters.

MutableOrderNewRequest orderRequest = makeMarketOrder(Side.BUY, 100, "AAPL");
submit (orderRequest);

Under the hood, this helper method initializes various fields of the OrderNewRequest to appropriate values, including generating a unique OrderID for the new order.

Order ID Generation Implementation

Order ID generation occurs within the OutboundOrderProcessor class. The method generateOrderId() handles the creation of new order IDs. By default, it is backed by a simple counter implementation that is sufficient for most use cases.

    protected MutableOrderNewRequest makeMarketOrder(Side side, @Decimal long quantity, CharSequence symbol) {
        MutableOrderNewRequest result = outboundOrderProcessor.makeOrder(side, quantity, symbol);
        result.setOrderType(OrderType.MARKET);
        return result;
    }

    class OutboundOrderProcessor {
        ...
        protected MutableOrderNewRequest makeOrder(Side side, @Decimal long quantity, CharSequence symbol) {
            MutableOrderNewRequest result = tradingMessages.getNewOrderRequest();
            result.setTimestamp(currentTime());
            result.setOrderId(generateOrderId()); <----------------- here we auto-generate a new order ID

            result.setSide(side);
            result.setQuantity(quantity);
            result.setSymbol(symbol);
            result.setSourceId(id);

            return result;
        }
    }

Since OrderID is a textual field, the CharSeqCounter class maintains the counter implementation. It stores the decimal number as an array of ASCII digits (ready to be viewed as a CharSequence) and provides an efficient way of incrementing the stored number.

public void increment() {  
    int i = WIDTH - 1 + offset;  
    while (true) {  
        byte c = buffer[i];  
        // increment character at index i  
        if (c < '9') {  
            buffer[i] = (byte) (c + 1);  
            break;  
        } else {  
            buffer[i] = '0';  
        }  
        if (--i == 0)  
            throw new ArithmeticException("Overflow");  
    }  
    start = Math.min(i - offset, start);  
}

Handling Duplicates After System Restart

To avoid order ID duplicates when the system restarts, the counter is initialized with the current time in microseconds.

CharSeqCounter counter = new CharSeqCounter(TimeUnit.MILLISECONDS.toMicros(epochClock.time()));

Duplicates Restart Example

Let's imagine the following situation:

Ember starts at T₀ and runs for 10 seconds, sending 100,000 orders per second from a single algorithm (usually 100x less). The last order ID would be T₀ + 1,000,000.

If a restart occurs in as little as 1 second (usually 30+ seconds), the next order ID would be T₀ + 15,000,000.

This method effectively prevents overlapping order ID sequences after system restarts.